{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 基于BiLSTM的中文文本多任务实战\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在我之前的博客中我们介绍了文本的多分类的方法,我们还尝试了各种分类模型,比如朴素贝叶斯、逻辑回归、支持向量机和随机森林等并且都取得了非常不错的效果。今天我们使用深度学习中的LSTM（Long Short-Term Memory）长短期记忆网络，它是一种时间循环神经网络，适合于处理和预测时间序列中间隔和延迟相对较长的重要事件。\n",
    "LSTM 已经在科技领域有了多种应用。基于 LSTM 的系统可以学习翻译语言、控制机器人、图像分析、文档摘要、语音识别图像识别、手写识别、控制聊天机器人、预测疾病、点击率和股票、合成音乐等等任务。今天我们用它来实现一下文本多分类，相信会取得较好的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数据\n",
    "\n",
    "我们的数据来自于互联网，你可以在这里下载,数据中包含了10 个类别（书籍、平板、手机、水果、洗发水、热水器、蒙牛、衣服、计算机、酒店），共 6 万多条评论数据 首先查看一下我们的数据，这些数据都是来自于电商网站的用户评价数据,我们想要把不同评价数据分到不同的分类中去,且每条数据只能对应10个类中的一个类。\n",
    "\n",
    "数据下载地址:https://github.com/SophonPlus/ChineseNlpCorpus/blob/master/datasets/online_shopping_10_cats/intro.ipynb\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#coding=utf-8\n",
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import matplotlib\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['font.sans-serif'] = ['SimHei']\n",
    "plt.rcParams['axes.unicode_minus'] = False\n",
    "\n",
    "from matplotlib.font_manager import FontProperties\n",
    "chinese_font = FontProperties(fname='/usr/share/fonts/SimHei.ttf')\n",
    "\n",
    "from keras.preprocessing.text import Tokenizer\n",
    "from keras.preprocessing.sequence import pad_sequences\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Embedding, LSTM, SpatialDropout1D\n",
    "from sklearn.model_selection import train_test_split\n",
    "from keras.utils.np_utils import to_categorical\n",
    "from keras.callbacks import EarlyStopping\n",
    "from keras.layers import Dropout\n",
    "from keras.layers import Input,LSTM,Bidirectional,Dense,Dropout,Concatenate,Embedding,GlobalMaxPool1D\n",
    "from keras.models import Model\n",
    "import keras.backend as K\n",
    "import jieba as jb\n",
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 62774 entries, 0 to 62773\n",
      "Data columns (total 3 columns):\n",
      "cat       62774 non-null object\n",
      "label     62774 non-null int64\n",
      "review    62773 non-null object\n",
      "dtypes: int64(1), object(2)\n",
      "memory usage: 1.4+ MB\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>label</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>书籍</td>\n",
       "      <td>1</td>\n",
       "      <td>﻿做父母一定要有刘墉这样的心态，不断地学习，不断地进步，不断地给自己补充新鲜血液，让自己保持...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>书籍</td>\n",
       "      <td>1</td>\n",
       "      <td>作者真有英国人严谨的风格，提出观点、进行论述论证，尽管本人对物理学了解不深，但是仍然能感受到...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>书籍</td>\n",
       "      <td>1</td>\n",
       "      <td>作者长篇大论借用详细报告数据处理工作和计算结果支持其新观点。为什么荷兰曾经县有欧洲最高的生产...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>书籍</td>\n",
       "      <td>1</td>\n",
       "      <td>作者在战几时之前用了＂拥抱＂令人叫绝．日本如果没有战败，就有会有美军的占领，没胡官僚主义的延...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>书籍</td>\n",
       "      <td>1</td>\n",
       "      <td>作者在少年时即喜阅读，能看出他精读了无数经典，因而他有一个庞大的内心世界。他的作品最难能可贵...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  cat  label                                             review\n",
       "0  书籍      1  ﻿做父母一定要有刘墉这样的心态，不断地学习，不断地进步，不断地给自己补充新鲜血液，让自己保持...\n",
       "1  书籍      1  作者真有英国人严谨的风格，提出观点、进行论述论证，尽管本人对物理学了解不深，但是仍然能感受到...\n",
       "2  书籍      1  作者长篇大论借用详细报告数据处理工作和计算结果支持其新观点。为什么荷兰曾经县有欧洲最高的生产...\n",
       "3  书籍      1  作者在战几时之前用了＂拥抱＂令人叫绝．日本如果没有战败，就有会有美军的占领，没胡官僚主义的延...\n",
       "4  书籍      1  作者在少年时即喜阅读，能看出他精读了无数经典，因而他有一个庞大的内心世界。他的作品最难能可贵..."
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('./data/online_shopping_10_cats.csv')\n",
    "df.info()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "数据总量: 62774 .\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>label</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>23124</th>\n",
       "      <td>水果</td>\n",
       "      <td>0</td>\n",
       "      <td>相比超市卖的，味道非常寡淡，而且烂了一个</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8991</th>\n",
       "      <td>平板</td>\n",
       "      <td>0</td>\n",
       "      <td>差评不好用，而且很卡，该店老板很小气，什么赠品都没有。上面写的正品都是骗人的，我一样都没有收...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36818</th>\n",
       "      <td>蒙牛</td>\n",
       "      <td>1</td>\n",
       "      <td>蒙牛的以前总喝，味道很不错</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16451</th>\n",
       "      <td>水果</td>\n",
       "      <td>1</td>\n",
       "      <td>包装可以，没有压坏的，还有水气从冰库拿出来的。口感还不错蛮好吃的，个头大小不一样，没有描述的...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8658</th>\n",
       "      <td>平板</td>\n",
       "      <td>1</td>\n",
       "      <td>特意等手机用了一周后来评价。发货贼快，上午下单，下午就到货。手机总体不错，适合看视频，京东网...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38115</th>\n",
       "      <td>蒙牛</td>\n",
       "      <td>0</td>\n",
       "      <td>蒙牛真不行！</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5235</th>\n",
       "      <td>平板</td>\n",
       "      <td>1</td>\n",
       "      <td>多快好省天天见，勤俭持家上京东。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40204</th>\n",
       "      <td>衣服</td>\n",
       "      <td>1</td>\n",
       "      <td>客服态度很好，裤子质量也不错，物流也快。很值的一次购物！！好评！！</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50743</th>\n",
       "      <td>计算机</td>\n",
       "      <td>1</td>\n",
       "      <td>小巧，做工还算精致。固态硬盘很超值，当然，这是在2999购买的时候。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47123</th>\n",
       "      <td>衣服</td>\n",
       "      <td>0</td>\n",
       "      <td>此物品不值得去买，如此垃圾，既然还有这么多好评，30块钱的货而已。</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       cat  label                                             review\n",
       "23124   水果      0                               相比超市卖的，味道非常寡淡，而且烂了一个\n",
       "8991    平板      0  差评不好用，而且很卡，该店老板很小气，什么赠品都没有。上面写的正品都是骗人的，我一样都没有收...\n",
       "36818   蒙牛      1                                      蒙牛的以前总喝，味道很不错\n",
       "16451   水果      1  包装可以，没有压坏的，还有水气从冰库拿出来的。口感还不错蛮好吃的，个头大小不一样，没有描述的...\n",
       "8658    平板      1  特意等手机用了一周后来评价。发货贼快，上午下单，下午就到货。手机总体不错，适合看视频，京东网...\n",
       "38115   蒙牛      0                                             蒙牛真不行！\n",
       "5235    平板      1                                   多快好省天天见，勤俭持家上京东。\n",
       "40204   衣服      1                  客服态度很好，裤子质量也不错，物流也快。很值的一次购物！！好评！！\n",
       "50743  计算机      1                 小巧，做工还算精致。固态硬盘很超值，当然，这是在2999购买的时候。\n",
       "47123   衣服      0                  此物品不值得去买，如此垃圾，既然还有这么多好评，30块钱的货而已。"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df[['cat','label','review']]\n",
    "print(\"数据总量: %d .\" % len(df))\n",
    "df.sample(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "在 cat 列中总共有 0 个空值.\n",
      "在 review 列中总共有 1 个空值.\n"
     ]
    }
   ],
   "source": [
    "print(\"在 cat 列中总共有 %d 个空值.\" % df['cat'].isnull().sum())\n",
    "print(\"在 review 列中总共有 %d 个空值.\" % df['review'].isnull().sum())\n",
    "df[df.isnull().values==True]\n",
    "df = df[pd.notnull(df['review'])]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>衣服</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>洗发水</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>水果</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>酒店</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>平板</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>计算机</td>\n",
       "      <td>3992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>书籍</td>\n",
       "      <td>3851</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>手机</td>\n",
       "      <td>2323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>蒙牛</td>\n",
       "      <td>2033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>热水器</td>\n",
       "      <td>574</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cat  count\n",
       "0   衣服  10000\n",
       "1  洗发水  10000\n",
       "2   水果  10000\n",
       "3   酒店  10000\n",
       "4   平板  10000\n",
       "5  计算机   3992\n",
       "6   书籍   3851\n",
       "7   手机   2323\n",
       "8   蒙牛   2033\n",
       "9  热水器    574"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = {'cat':df['cat'].value_counts().index, 'count': df['cat'].value_counts()}\n",
    "df_cat = pd.DataFrame(data=d).reset_index(drop=True)\n",
    "df_cat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,0,'类目')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/matplotlib/font_manager.py:1320: UserWarning: findfont: Font family ['sans-serif'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAFSCAYAAABv4dyYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAF5dJREFUeJzt3X+wJWV95/H3R0YQYfkVJkQHcHAhpsCsi0yQXcrESIQRk8WtIEUSw8QiTpWgkZQljqktx0JJoWtELBddEkjQNSJF3IJElCBIpdYsxDEYAygyy8+ZRRnlh1ksg0O++8d5pjxM7uXOj3vOvX2f96vq1O1++uk+3z7TM/cz3U/3SVUhSZL69ZyFLkCSJC0sw4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxI2mlJMqX3OTHJ3tN4L6lnhgFJzyrJxUlemOTXk7wjyb7Azdt+SSd5Y5JvJ/nSdq97k/zmDNv7nST7Jlmd5JRned99gf8B7DGxnZMEQHzokKTZJFkG/ALwZuA6YCVwMHAL8GXgX4A3AIdX1Ye2W/e9wF1VdfVY24uAzwGrgJ8G/gp4ZVX9qC0/FXg38DSwP7ACuGtss88BLqmqv5jfPZX6ZhiQNKMkBzH6xf0ksDfwy4x+MT8JPALsCbwFOBb4Q+DB7TbxYuD8qrpmbJt/CXysqm5o828HfhE4s6p+nOQ5jH7hPxf438CrgJP85S9NlmFA0rNK8krgncDtwBPAC4DlwHuq6sEkpwMrZzkzcMe2MJDk9xkFhzU19g9Pkg8y+qX/u1X1jdZ2MXBPVV2a5JaqetVk91Lq27KFLkDS4pXkEkb/4z8TeCXw8qp6Z5J/B5wDrNvB7bwIeDXwbeCOJAcD9/GT8QAfA/Zpfd/ctn13kjOAlyW5BdgXuK2qzp2n3ZPUGAYkzSjJMcDPA0cyukTwM8ABSU5sXfZu1/gB1iZZvd0mXgycD1BVDwC/1rb7FeClVXVBCwkfrqor27LfAn4H+CDwlar6YpIvVdWvJPn3bZmkeWYYkDSjqrqT0f/mSbIX8EVGZwn+qKq+sq1fu0xw2SyXCWbyRuADbfqFwENjy/5ne50zD7sgaQcZBiTNKsnzgFOAtwEXALcCn2in7/+4qu4AwuxnBt613fZ+D9haVbe1pp9lLAxU1Q9bv73GVtsjydHAfwW+MF/7JuknDAOSZtSu638W+Gvg9Kp6vC1ak+Q1wIVJzmP078ilVfWR7dZ/b1u27SFFnwO+z+jMAEnWA68Bzprh7f8c+EGb3g/4P4wuOXx9vvZP0k94N4GkqUiyT1U9udB1SPrXDAOSJHXOxxFLktQ5w4AkSZ3ragDhwQcfXCtXrlzoMiRJmoqvfe1r36uq5XP16yoMrFy5kg0bNix0GZIkTUWSB3akn5cJJEnqnGFAkqTOGQYkSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOTSQMJLkiySNJ7hhrOyjJjUnuaT8PbO1J8tEkG5N8I8nLx9ZZ0/rfk2TNWPtxSf6xrfPR9vWokiRpF0zqzMCfAau3a1sH3FRVRwE3tXmA1wJHtdda4OMwCg/AeuAVwPHA+m0BovV589h627+XJEnaQRMJA1X1N8Cj2zWfBlzZpq8EXj/W/skauRU4IMkLgFOAG6vq0ap6DLgRWN2W7VdVt9bo+5c/ObYtSZK0k6b53QSHVNXDbfo7wCFtegXw0Fi/Ta3t2do3zdA+oyRrGZ1x4PDDD9/hYleu+/wO990Z91/0uolsd1L1wvBqnlS9MLyah3hcSJq+BRlA2P5HX1N6r8uqalVVrVq+fM4vbpIkqTvTDAPfbaf4aT8fae2bgcPG+h3a2p6t/dAZ2iVJ0i6YZhi4Dth2R8Aa4Nqx9rPaXQUnAE+0ywk3ACcnObANHDwZuKEt+0GSE9pdBGeNbUuSJO2kiYwZSPIZ4FXAwUk2Mbor4CLg6iRnAw8AZ7Tu1wOnAhuBHwJvAqiqR5O8D/hq63dBVW0blHgOozsW9ga+0F6SJGkXTCQMVNVvzLLopBn6FnDuLNu5ArhihvYNwEt3p0ZJkjTiEwglSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOGQYkSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOGQYkSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOGQYkSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOGQYkSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOGQYkSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOGQYkSeqcYUCSpM4ZBiRJ6pxhQJKkzhkGJEnqnGFAkqTOTT0MJPn9JHcmuSPJZ5I8L8kRSW5LsjHJZ5Ps2fru1eY3tuUrx7bz7tZ+d5JTpr0fkiQtFVMNA0lWAL8HrKqqlwJ7AGcCHwAurqojgceAs9sqZwOPtfaLWz+SHN3WOwZYDVyaZI9p7oskSUvFQlwmWAbsnWQZ8HzgYeDVwDVt+ZXA69v0aW2etvykJGntV1XVP1fVfcBG4Pgp1S9J0pIy1TBQVZuBDwEPMgoBTwBfAx6vqq2t2yZgRZteATzU1t3a+v/UePsM60iSpJ0w7csEBzL6X/0RwAuBfRid5p/ke65NsiHJhi1btkzyrSRJGqRpXyb4FeC+qtpSVT8GPgecCBzQLhsAHApsbtObgcMA2vL9ge+Pt8+wzjNU1WVVtaqqVi1fvny+90eSpMGbdhh4EDghyfPbtf+TgLuALwOntz5rgGvb9HVtnrb85qqq1n5mu9vgCOAo4O+mtA+SJC0py+buMn+q6rYk1wB/D2wFbgcuAz4PXJXk/a3t8rbK5cCnkmwEHmV0BwFVdWeSqxkFia3AuVX19DT3RZKkpWKqYQCgqtYD67drvpcZ7gaoqh8Bb5hlOxcCF857gZIkdcYnEEqS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1Lmph4EkByS5Jsm3knwzyX9IclCSG5Pc034e2PomyUeTbEzyjSQvH9vOmtb/niRrpr0fkiQtFQtxZuAS4ItV9XPAy4BvAuuAm6rqKOCmNg/wWuCo9loLfBwgyUHAeuAVwPHA+m0BQpIk7ZyphoEk+wO/CFwOUFVPVdXjwGnAla3blcDr2/RpwCdr5FbggCQvAE4BbqyqR6vqMeBGYPUUd0WSpCVj2mcGjgC2AH+a5PYkf5JkH+CQqnq49fkOcEibXgE8NLb+ptY2W/u/kmRtkg1JNmzZsmUed0WSpKVh2mFgGfBy4ONVdSzwJD+5JABAVRVQ8/WGVXVZVa2qqlXLly+fr81KkrRkTDsMbAI2VdVtbf4aRuHgu+30P+3nI235ZuCwsfUPbW2ztUuSpJ001TBQVd8BHkryktZ0EnAXcB2w7Y6ANcC1bfo64Kx2V8EJwBPtcsINwMlJDmwDB09ubZIkaSctW4D3fBvw6SR7AvcCb2IUSq5OcjbwAHBG63s9cCqwEfhh60tVPZrkfcBXW78LqurR6e2CJElLx9TDQFV9HVg1w6KTZuhbwLmzbOcK4Ir5rU6SpP74BEJJkjpnGJAkqXOGAUmSOmcYkCSpc4YBSZI6ZxiQJKlzhgFJkjq3w2EgySkztB03v+VIkqRpe9aHDiV5IfA08GPg7UnuB/YAfgAcB7wVeM2Ea5QkSRM01xMINzD6BsE/ZxQAPsjoi4X+FPgl4ImJVidJkiZurjBwN6Mw8C1gBfA3wH7A94B/mWxpkiRpGnb0uwkKCHAU8NOMLhEcAjx/QnVJkqQp2dW7CWrsJUmSBmxnw8C3ge8Cfw880l6SJGnA5rpM8HPt5zGMLhP8EvCzwMGM7iqQJEkDN1cYOA54itHthZ8BzmN0NuGfgHuBN060OkmSNHHPGgaq6v8CJFkFfLqqvjW2+M+S/MMki5MkSZM355iBJAGuBY5N8tYkv9DajwMunXB9kiRpwuZ6AmGqqpJsBP478CLgtUk+wejywRumUKMkSZqgucYMfCHJ/wP2B45kNIZgFfCXjAYVLgc2TbRCSZI0UXOFgdOBlcC5wAXAw8BpVfV0kpXAFUlOqiqfNyBJ0kDNFQbOYXRG4PvAN4CPAPcl+TTwUuBdBgFJkoZtrgGE+7Wf/xHYi1F4+Afg64zOGNw5scokSdJUzBUGbgTuA45l9GVF/xl4GbCa0TcY/sFEq5MkSRM3Vxh4NaO7Bj4GvAT4FLARuLWqPgX82yS7+v0GkiRpEZjroUPrk+zD6G6Cp1v//1JVf9u6vKWq/CpjSZIGbM6vMK6qJ4Enx5o2jy17fBJFSZKk6fEUvyRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktS5BQkDSfZIcnuSv2rzRyS5LcnGJJ9Nsmdr36vNb2zLV45t492t/e4kpyzEfkiStBQs1JmBtwPfHJv/AHBxVR0JPAac3drPBh5r7Re3fiQ5GjgTOAZYDVyaZI8p1S5J0pIy9TCQ5FDgdcCftPkArwauaV2uBF7fpk9r87TlJ7X+pwFXVdU/V9V9wEbg+OnsgSRJS8uyBXjPjwDnA/+mzf8U8HhVbW3zm4AVbXoF8BBAVW1N8kTrvwK4dWyb4+s8Q5K1wFqAww8/fP72QtJErFz3+Yls9/6LXjeR7UpLwVTPDCT5VeCRqvratN6zqi6rqlVVtWr58uXTeltJkgZj2mcGTgT+U5JTgecB+wGXAAckWdbODhwKbG79NwOHAZuSLAP2B74/1r7N+DqSJGknTPXMQFW9u6oOraqVjAYA3lxVvwV8GTi9dVsDXNumr2vztOU3V1W19jPb3QZHAEcBfzel3ZAkaUlZiDEDM3kXcFWS9wO3A5e39suBTyXZCDzKKEBQVXcmuRq4C9gKnFtVT0+/bElynIOGb8HCQFXdAtzSpu9lhrsBqupHwBtmWf9C4MLJVShJUh98AqEkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmdMwxIktQ5w4AkSZ0zDEiS1LllC12AJGm6Vq77/MS2ff9Fr5vYtjU5nhmQJKlzhgFJkjpnGJAkqXOGAUmSOjfVMJDksCRfTnJXkjuTvL21H5TkxiT3tJ8HtvYk+WiSjUm+keTlY9ta0/rfk2TNNPdDkqSlZNpnBrYC76iqo4ETgHOTHA2sA26qqqOAm9o8wGuBo9prLfBxGIUHYD3wCuB4YP22ACFJknbOVG8trKqHgYfb9D8l+SawAjgNeFXrdiVwC/Cu1v7Jqirg1iQHJHlB63tjVT0KkORGYDXwmantjCRpaiZ1O6S3Qo4s2JiBJCuBY4HbgENaUAD4DnBIm14BPDS22qbWNlv7TO+zNsmGJBu2bNkyb/VLkrRULEgYSLIv8BfAeVX1g/Fl7SxAzdd7VdVlVbWqqlYtX758vjYrSdKSMfUwkOS5jILAp6vqc635u+30P+3nI619M3DY2OqHtrbZ2iVJ0k6a9t0EAS4HvllVHx5bdB2w7Y6ANcC1Y+1ntbsKTgCeaJcTbgBOTnJgGzh4cmuTJEk7adrfTXAi8NvAPyb5emv7A+Ai4OokZwMPAGe0ZdcDpwIbgR8CbwKoqkeTvA/4aut3wbbBhJIkaedM+26C/wVklsUnzdC/gHNn2dYVwBXzV50kSX3yCYSSJHXOMCBJUucMA5Ikdc4wIElS5wwDkiR1zjAgSVLnDAOSJHXOMCBJUucMA5Ikdc4wIElS5wwDkiR1zjAgSVLnDAOSJHXOMCBJUucMA5Ikdc4wIElS5wwDkiR1zjAgSVLnDAOSJHXOMCBJUucMA5Ikdc4wIElS5wwDkiR1zjAgSVLnDAOSJHXOMCBJUucMA5Ikdc4wIElS5wwDkiR1zjAgSVLnDAOSJHVu2UIXIEnSUrRy3ecnst37L3rdvG/TMwOSJHXOMCBJUucMA5Ikdc4wIElS5wwDkiR1zjAgSVLnDAOSJHXOMCBJUucMA5Ikdc4wIElS5wYdBpKsTnJ3ko1J1i10PZIkDdFgw0CSPYD/BrwWOBr4jSRHL2xVkiQNz2DDAHA8sLGq7q2qp4CrgNMWuCZJkgZnyGFgBfDQ2Pym1iZJknZCqmqha9glSU4HVlfV77b53wZeUVVv3a7fWmBtm30JcPcEyjkY+N4EtjtJQ6t5aPWCNU/D0OqF4dU8tHrBmse9qKqWz9Vp2QTeeFo2A4eNzR/a2p6hqi4DLptkIUk2VNWqSb7HfBtazUOrF6x5GoZWLwyv5qHVC9a8K4Z8meCrwFFJjkiyJ3AmcN0C1yRJ0uAM9sxAVW1N8lbgBmAP4IqqunOBy5IkaXAGGwYAqup64PqFroMJX4aYkKHVPLR6wZqnYWj1wvBqHlq9YM07bbADCCVJ0vwY8pgBSZI0DwwDkiR1zjAgSVLnBj2AcKEkec8cXR6pqk9MpZgdZM2TN7R6wZqnYWj1wvBqHlq9sPhqNgzsmhMYPdcgsyy/ElhUBx7WPA1DqxeseRqGVi8Mr+ah1QuLrGbDwK55uqp+MNvCJIvxFg1rnryh1QvWPA1DqxeGV/PQ6oVFVrNjBnbNXH9Ii/HAs+bJG1q9YM3TMLR6YXg1D61eWGQ1e2Zg1zw3yX6zLAujJyIuNtY8eUOrF6x5GoZWLwyv5qHVC4usZsPArrkVOG+WZQG+MMVadtSz1QzDq3kxfs5DqxeWXs2w+Gpeap/xYqx5aPXCIjuODQO75hUsooEfO2G2eheroX3OQ6sXhlkzDOtYHuJnPLSah1bvNovmODYM7JpFNfBjBw3xL8vQPueh1QvDrHlox/IQP+Oh1Ty0emGRHceGgV2zqAZ+7KAh/mUZ2uc8tHphmDUP7Vge4mc8tJqHVi8ssuPYMLBrFtXAjx00xL8sQ/uch1YvDLPmoR3LQ/yMh1bz0OqFRXYcGwZ2zbaBH7Od3vniFGvZUUP8yzK0z3lo9cIwax7asTzEz3hoNQ+tXlhkx7FfYdyJJOuZPWkG+O5ie1ynNBOPZS0Fi+049sxAPxbVYBVpN3gsaylYVMexYaAfi2qwirQbPJa1FCyq49jHEfdjUQ1WkXaDx7KWgkV1HHtmoB+LarCKtBs8lrUULKrj2DDQjyGOtpVm4rGspWBRHcfeTSBJUuccMyBJUucMA5ImJsmi+SIWSbMzDEiaF0kuTvLCJL+e5B1J9gVuTrJ3W/7GJN9O8qXtXvcm+c0FLl/qmgMIJe22JMuAq4H3A9cBTwPrgAuBp9oZgqeAy6rqQ9ut+15g61QLlvQMDiCUtFuSHAR8DngS2Bv4ZeCuNv8IsCfwFuBY4A+BB7fbxIuB86vqmmnVLOmZDAOS5kWSVwLvBG4HngBeACwH3lNVDyY5HVg5y5mBOwwD0sJxzICk3ZbkEuB4Rs9a/1tgr6p6J/Bh4JyFrE3S3BwzIGm3JDkG+HngSEaXCH4GOCDJia3L3klObdNrk6zebhMvBs6fSrGSZuRlAknzJslejJ6c9hjwR1X1lbFlXiaQFinPDEjabUmeB5wCvA24gNGjVj+R5Azgj6vqDkaPXZ3tzMC7plmvpGcyDEjaLUkOBj4L/DVwelU93hatSfIa4MIk5zH69+bSqvrIduu/F/8tkhaUlwkkSeqcdxNIktQ5w4AkSZ0zDEiS1DnDgCRJnTMMSJLUOcOAJEmd+/8ZsXuZzabWGAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f796cd1c6a0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "df_cat.plot(x='cat', y='count', kind='bar', legend=False,  figsize=(8, 5))\n",
    "plt.title(\"类目分布\", fontproperties=chinese_font)\n",
    "plt.ylabel('数量', fontsize=18, fontproperties=chinese_font)\n",
    "plt.xlabel('类目', fontsize=18, fontproperties=chinese_font)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数据预处理\n",
    "\n",
    " 接下来我们要将cat类转换成id，这样便于以后的分类模型的训练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>label</th>\n",
       "      <th>review</th>\n",
       "      <th>cat_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>52395</th>\n",
       "      <td>计算机</td>\n",
       "      <td>0</td>\n",
       "      <td>删除LIUNX系统有点麻烦。屏幕太小分辨率只有 1024*600看都不舒服。</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49642</th>\n",
       "      <td>计算机</td>\n",
       "      <td>1</td>\n",
       "      <td>外观不错，XP的驱动官网上都有，用了两个小时重装了xp，基本上速度没问题了 音质不错</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7395</th>\n",
       "      <td>平板</td>\n",
       "      <td>1</td>\n",
       "      <td>嗯，还行，用了快二星期，没有出现问题(O_O，什么白班还有重影的都不是问题，你既然买了，就用...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62635</th>\n",
       "      <td>酒店</td>\n",
       "      <td>0</td>\n",
       "      <td>吹风机太烂了，根本没法用</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53401</th>\n",
       "      <td>酒店</td>\n",
       "      <td>1</td>\n",
       "      <td>服务，房间都很不错。交通也不错。酒店附近就有很多小饭店，都还不错的。12路公交一直到火车站，...</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5116</th>\n",
       "      <td>平板</td>\n",
       "      <td>1</td>\n",
       "      <td>黑边大，其他的还算满意，就底盖圆角位置有小小脱胶</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15390</th>\n",
       "      <td>手机</td>\n",
       "      <td>0</td>\n",
       "      <td>信号不是很好，有自动关机的现象！做工不好，感觉翻盖不流畅，转轴处还易积灰，很难清理。拍出的照...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30647</th>\n",
       "      <td>洗发水</td>\n",
       "      <td>1</td>\n",
       "      <td>'满满的黄瓜味和我三年前买的完全不一样。妈*这两天和媳妇打架可能要离婚他*的！</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5083</th>\n",
       "      <td>平板</td>\n",
       "      <td>1</td>\n",
       "      <td>物流神速，手机还没用，用后再评</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3716</th>\n",
       "      <td>书籍</td>\n",
       "      <td>0</td>\n",
       "      <td>不喜欢,这样的 写的方式,看大家都买,所以买了,晕啊!</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       cat  label                                             review  cat_id\n",
       "52395  计算机      0             删除LIUNX系统有点麻烦。屏幕太小分辨率只有 1024*600看都不舒服。       8\n",
       "49642  计算机      1         外观不错，XP的驱动官网上都有，用了两个小时重装了xp，基本上速度没问题了 音质不错       8\n",
       "7395    平板      1  嗯，还行，用了快二星期，没有出现问题(O_O，什么白班还有重影的都不是问题，你既然买了，就用...       1\n",
       "62635   酒店      0                                       吹风机太烂了，根本没法用       9\n",
       "53401   酒店      1  服务，房间都很不错。交通也不错。酒店附近就有很多小饭店，都还不错的。12路公交一直到火车站，...       9\n",
       "5116    平板      1                           黑边大，其他的还算满意，就底盖圆角位置有小小脱胶       1\n",
       "15390   手机      0  信号不是很好，有自动关机的现象！做工不好，感觉翻盖不流畅，转轴处还易积灰，很难清理。拍出的照...       2\n",
       "30647  洗发水      1            '满满的黄瓜味和我三年前买的完全不一样。妈*这两天和媳妇打架可能要离婚他*的！       4\n",
       "5083    平板      1                                    物流神速，手机还没用，用后再评       1\n",
       "3716    书籍      0                        不喜欢,这样的 写的方式,看大家都买,所以买了,晕啊!       0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['cat_id'] = df['cat'].factorize()[0]\n",
    "cat_id_df = df[['cat', 'cat_id']].drop_duplicates().sort_values('cat_id').reset_index(drop=True)\n",
    "cat_to_id = dict(cat_id_df.values)\n",
    "id_to_cat = dict(cat_id_df[['cat_id', 'cat']].values)\n",
    "df.sample(10)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>cat_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>书籍</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>平板</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>手机</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>水果</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>洗发水</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>热水器</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>蒙牛</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>衣服</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>计算机</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>酒店</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cat  cat_id\n",
       "0   书籍       0\n",
       "1   平板       1\n",
       "2   手机       2\n",
       "3   水果       3\n",
       "4  洗发水       4\n",
       "5  热水器       5\n",
       "6   蒙牛       6\n",
       "7   衣服       7\n",
       "8  计算机       8\n",
       "9   酒店       9"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat_id_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们将cat转换成了Id(0到9),由于我们的评价内容都是中文,所以要对中文进行一些预处理工作,这包括删除文本中的标点符号,特殊符号,还要删除一些无意义的常用词(stopword),因为这些词和符号对系统分析预测文本的内容没有任何帮助,反而会增加计算的复杂度和增加系统开销,所有在使用这些文本数据之前必须要将它们清理干净。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#定义删除除字母,数字，汉字以外的所有符号的函数\n",
    "def remove_punctuation(line):\n",
    "    line = str(line)\n",
    "    if line.strip()=='':\n",
    "        return ''\n",
    "    rule = re.compile(u\"[^a-zA-Z0-9\\u4E00-\\u9FA5]\")\n",
    "    line = rule.sub('',line)\n",
    "    return line\n",
    " \n",
    "def stopwordslist(filepath):  \n",
    "    stopwords = [line.strip() for line in open(filepath, 'r', encoding='utf-8').readlines()]  \n",
    "    return stopwords  \n",
    "\n",
    "#加载停用词\n",
    "stopwords = stopwordslist(\"./data/chineseStopWords.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "中文停用词包含了很多日常使用频率很高的常用词,如 吧，吗，呢，啥等一些感叹词等,这些高频常用词无法反应出文本的主要意思,所以要被过滤掉。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>label</th>\n",
       "      <th>review</th>\n",
       "      <th>cat_id</th>\n",
       "      <th>clean_review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33887</th>\n",
       "      <td>洗发水</td>\n",
       "      <td>0</td>\n",
       "      <td>没有收到货物居然给我回复是已签收，快递员联系不上，什么原因？</td>\n",
       "      <td>4</td>\n",
       "      <td>没有收到货物居然给我回复是已签收快递员联系不上什么原因</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35882</th>\n",
       "      <td>洗发水</td>\n",
       "      <td>0</td>\n",
       "      <td>差评，一起买的纸巾居然没经过我同意就自动退货了</td>\n",
       "      <td>4</td>\n",
       "      <td>差评一起买的纸巾居然没经过我同意就自动退货了</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22690</th>\n",
       "      <td>水果</td>\n",
       "      <td>0</td>\n",
       "      <td>发货速度慢 ，没想象中的好 还有些坏掉了</td>\n",
       "      <td>3</td>\n",
       "      <td>发货速度慢没想象中的好还有些坏掉了</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22754</th>\n",
       "      <td>水果</td>\n",
       "      <td>0</td>\n",
       "      <td>三个果，只有一个是好的。一个轻度腐烂，一个重度腐烂。</td>\n",
       "      <td>3</td>\n",
       "      <td>三个果只有一个是好的一个轻度腐烂一个重度腐烂</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55683</th>\n",
       "      <td>酒店</td>\n",
       "      <td>1</td>\n",
       "      <td>很不错的酒店，在海宁皮革城对面，遗憾的是到酒店很晚了，没有去逛。早餐也不错，去的时候正好赶到...</td>\n",
       "      <td>9</td>\n",
       "      <td>很不错的酒店在海宁皮革城对面遗憾的是到酒店很晚了没有去逛早餐也不错去的时候正好赶到FIBA在...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45421</th>\n",
       "      <td>衣服</td>\n",
       "      <td>0</td>\n",
       "      <td>样子很好，只是布料不咋地</td>\n",
       "      <td>7</td>\n",
       "      <td>样子很好只是布料不咋地</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46091</th>\n",
       "      <td>衣服</td>\n",
       "      <td>0</td>\n",
       "      <td>这裤子太坑了</td>\n",
       "      <td>7</td>\n",
       "      <td>这裤子太坑了</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49365</th>\n",
       "      <td>计算机</td>\n",
       "      <td>1</td>\n",
       "      <td>相对其他品牌来说我更喜欢小黑，磨砂的壳，很有质感，不容易留指纹，散热就不用说了，很ok的！预...</td>\n",
       "      <td>8</td>\n",
       "      <td>相对其他品牌来说我更喜欢小黑磨砂的壳很有质感不容易留指纹散热就不用说了很ok的预装的VIST...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57458</th>\n",
       "      <td>酒店</td>\n",
       "      <td>1</td>\n",
       "      <td>酒店服务真的很好，房间也很整洁。从阳台可以看到海景，海上有好多停着的船只，蛮有感觉的。自助早...</td>\n",
       "      <td>9</td>\n",
       "      <td>酒店服务真的很好房间也很整洁从阳台可以看到海景海上有好多停着的船只蛮有感觉的自助早餐很不错品...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30627</th>\n",
       "      <td>洗发水</td>\n",
       "      <td>1</td>\n",
       "      <td>很不错，之前老听说这洗发水好，试试。用完回复朋友们</td>\n",
       "      <td>4</td>\n",
       "      <td>很不错之前老听说这洗发水好试试用完回复朋友们</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       cat  label                                             review  cat_id  \\\n",
       "33887  洗发水      0                     没有收到货物居然给我回复是已签收，快递员联系不上，什么原因？       4   \n",
       "35882  洗发水      0                            差评，一起买的纸巾居然没经过我同意就自动退货了       4   \n",
       "22690   水果      0                               发货速度慢 ，没想象中的好 还有些坏掉了       3   \n",
       "22754   水果      0                         三个果，只有一个是好的。一个轻度腐烂，一个重度腐烂。       3   \n",
       "55683   酒店      1  很不错的酒店，在海宁皮革城对面，遗憾的是到酒店很晚了，没有去逛。早餐也不错，去的时候正好赶到...       9   \n",
       "45421   衣服      0                                       样子很好，只是布料不咋地       7   \n",
       "46091   衣服      0                                             这裤子太坑了       7   \n",
       "49365  计算机      1  相对其他品牌来说我更喜欢小黑，磨砂的壳，很有质感，不容易留指纹，散热就不用说了，很ok的！预...       8   \n",
       "57458   酒店      1  酒店服务真的很好，房间也很整洁。从阳台可以看到海景，海上有好多停着的船只，蛮有感觉的。自助早...       9   \n",
       "30627  洗发水      1                          很不错，之前老听说这洗发水好，试试。用完回复朋友们       4   \n",
       "\n",
       "                                            clean_review  \n",
       "33887                        没有收到货物居然给我回复是已签收快递员联系不上什么原因  \n",
       "35882                             差评一起买的纸巾居然没经过我同意就自动退货了  \n",
       "22690                                  发货速度慢没想象中的好还有些坏掉了  \n",
       "22754                             三个果只有一个是好的一个轻度腐烂一个重度腐烂  \n",
       "55683  很不错的酒店在海宁皮革城对面遗憾的是到酒店很晚了没有去逛早餐也不错去的时候正好赶到FIBA在...  \n",
       "45421                                        样子很好只是布料不咋地  \n",
       "46091                                             这裤子太坑了  \n",
       "49365  相对其他品牌来说我更喜欢小黑磨砂的壳很有质感不容易留指纹散热就不用说了很ok的预装的VIST...  \n",
       "57458  酒店服务真的很好房间也很整洁从阳台可以看到海景海上有好多停着的船只蛮有感觉的自助早餐很不错品...  \n",
       "30627                             很不错之前老听说这洗发水好试试用完回复朋友们  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#删除除字母,数字，汉字以外的所有符号\n",
    "df['clean_review'] = df['review'].apply(remove_punctuation)\n",
    "df.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们过滤掉了review中的标点符号和一些特殊符号，并生成了一个新的字段 clean_review。接下来我们要在clean_review的基础上进行分词,把每个评论内容分成由空格隔开的一个一个单独的词语。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Building prefix dict from the default dictionary ...\n",
      "Loading model from cache /tmp/jieba.cache\n",
      "Loading model cost 0.494 seconds.\n",
      "Prefix dict has been built succesfully.\n"
     ]
    }
   ],
   "source": [
    "#分词，并过滤停用词\n",
    "df['cut_review'] = df['clean_review'].apply(lambda x: \" \".join([w for w in list(jb.cut(x)) if w not in stopwords]))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>cat_id</th>\n",
       "      <th>label</th>\n",
       "      <th>cut_review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16587</th>\n",
       "      <td>水果</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>翠翠 口感 不错 送来 带 水珠</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3561</th>\n",
       "      <td>书籍</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>中医 穴位 好象 有用 其实不然 知道 这书 红 起来 使用 性 很大</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20680</th>\n",
       "      <td>水果</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>脆甜 微酸 个头 很大 汁水 饱满 口感 蛮</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18591</th>\n",
       "      <td>水果</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>物流 快 服务态度 水果 不错 下次 买</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27277</th>\n",
       "      <td>洗发水</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>一如既往 快 态度 日期 新</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       cat  cat_id  label                           cut_review\n",
       "16587   水果       3      1                     翠翠 口感 不错 送来 带 水珠\n",
       "3561    书籍       0      0  中医 穴位 好象 有用 其实不然 知道 这书 红 起来 使用 性 很大\n",
       "20680   水果       3      1               脆甜 微酸 个头 很大 汁水 饱满 口感 蛮\n",
       "18591   水果       3      1                 物流 快 服务态度 水果 不错 下次 买\n",
       "27277  洗发水       4      1                       一如既往 快 态度 日期 新"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df[['cat', 'cat_id', 'label', 'cut_review']]\n",
    "df.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BiLSTM建模 \n",
    "\n",
    "数据预处理完成以后，接下来我们要开始进行BiLSTM的建模工作：\n",
    "\n",
    "* 我们要将cut_review数据进行向量化处理,我们要将每条cut_review转换成一个整数序列的向量\n",
    "* 设置最频繁使用的50000个词\n",
    "* 设置每条 cut_review最大的词语数为250个(超过的将会被截去,不足的将会被补0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "共有 70364 个不相同的词语.\n"
     ]
    }
   ],
   "source": [
    "# 设置最频繁使用的50000个词(在texts_to_matrix是会取前MAX_NB_WORDS,会取前MAX_NB_WORDS列)\n",
    "MAX_NB_WORDS = 50000\n",
    "# 每条cut_review最大的长度\n",
    "MAX_SEQUENCE_LENGTH = 250\n",
    "# 设置Embeddingceng层的维度\n",
    "EMBEDDING_DIM = 100\n",
    "\n",
    "tokenizer = Tokenizer(num_words=MAX_NB_WORDS, filters='!\"#$%&()*+,-./:;<=>?@[\\]^_`{|}~', lower=True)\n",
    "tokenizer.fit_on_texts(df['cut_review'].values)\n",
    "word_index = tokenizer.word_index\n",
    "print('共有 %s 个不相同的词语.' % len(word_index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练和测试的数据集都准备好以后,接下来我们要定义一个LSTM的序列模型:  \n",
    "\n",
    "* 模型的第一次是嵌入层(Embedding)，它使用长度为100的向量来表示每一个词语\n",
    "* SpatialDropout1D层在训练中每次更新时， 将输入单元的按比率随机设置为 0， 这有助于防止过拟合\n",
    "* LSTM层包含100个记忆单元\n",
    "* 输出层为包含10个分类的全连接层\n",
    "* 由于是多分类，所以激活函数设置为'softmax'\n",
    "* 由于是多分类, 所以损失函数为分类交叉熵categorical_crossentropy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = tokenizer.texts_to_sequences(df['cut_review'].values)\n",
    "#填充X,让X的各个列的长度统一\n",
    "X = pad_sequences(X, maxlen=MAX_SEQUENCE_LENGTH)\n",
    "\n",
    "#多类标签的onehot展开\n",
    "Y0 = pd.get_dummies(df['cat_id']).values\n",
    "Y1 = pd.get_dummies(df['label']).values\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(62773, 12)\n",
      "[[1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]\n",
      " [1 0 0 0 0 0 0 0 0 0 0 1]]\n"
     ]
    }
   ],
   "source": [
    "Y = np.concatenate((Y0, Y1),axis=1) \n",
    "#Y = np.array(list(map(lambda x:np.array(x), zip(Y0.tolist(), Y1.tolist()))),dtype=object)\n",
    "\n",
    "print(Y.shape)\n",
    "print(Y[:10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(56495, 250) (56495, 12)\n",
      "(6278, 250) (6278, 12)\n"
     ]
    }
   ],
   "source": [
    "#拆分训练集和测试集\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.10, random_state = 42)\n",
    "print(X_train.shape,Y_train.shape)\n",
    "print(X_test.shape,Y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "Model: \"model_1\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "sen_emb (InputLayer)            (None, 250)          0                                            \n",
      "__________________________________________________________________________________________________\n",
      "embedding_1 (Embedding)         (None, 250, 100)     5000000     sen_emb[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "spatial_dropout1d_1 (SpatialDro (None, 250, 100)     0           embedding_1[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "bidirectional_1 (Bidirectional) (None, 200)          160800      spatial_dropout1d_1[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "out1 (Dense)                    (None, 10)           2010        bidirectional_1[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "out2 (Dense)                    (None, 2)            402         bidirectional_1[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_1 (Concatenate)     (None, 12)           0           out1[0][0]                       \n",
      "                                                                 out2[0][0]                       \n",
      "==================================================================================================\n",
      "Total params: 5,163,212\n",
      "Trainable params: 5,163,212\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "\n",
    "#定义模型\n",
    "K.clear_session()\n",
    "inputs = Input(shape=(MAX_SEQUENCE_LENGTH, ), name=\"sen_emb\")\n",
    "emb = Embedding(MAX_NB_WORDS, EMBEDDING_DIM, input_length=MAX_SEQUENCE_LENGTH)(inputs)\n",
    "x = SpatialDropout1D(0.2)(emb)\n",
    "x = Bidirectional(LSTM(100, dropout=0.2, recurrent_dropout=0.2))(x)\n",
    "\n",
    "###任务一，10分类的文本分类任务\n",
    "out1 = Dense(10, activation='softmax', name=\"out1\")(x)\n",
    "###任务二，情感分类\n",
    "out2 = Dense(2, activation='softmax', name=\"out2\")(x)\n",
    "\n",
    "# 把两个输出连起来\n",
    "out = Concatenate(axis=1)([out1, out2])\n",
    "model = Model(inputs=[inputs], outputs=[out])\n",
    "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])  # binary_crossentropy categorical_crossentropy\n",
    "\n",
    "\n",
    "#model = Model(inputs=[inputs], outputs=[out1, out2]) \n",
    "#model.compile(optimizer='adam', \n",
    "#              loss={'out1': 'categorical_crossentropy','out2': 'categorical_crossentropy'},\n",
    "#              loss_weights={'out1':1, 'out2': 1},\n",
    "#              metrics=[\"acc\"])\n",
    "\n",
    "model.summary()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "WARNING:tensorflow:From /home/hexi/.local/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n",
      "\n",
      "Train on 50845 samples, validate on 5650 samples\n",
      "Epoch 1/3\n",
      "50845/50845 [==============================] - 355s 7ms/step - loss: 0.1892 - accuracy: 0.9260 - val_loss: 0.1023 - val_accuracy: 0.9614\n",
      "Epoch 2/3\n",
      "50845/50845 [==============================] - 354s 7ms/step - loss: 0.0871 - accuracy: 0.9677 - val_loss: 0.0865 - val_accuracy: 0.9677\n",
      "Epoch 3/3\n",
      "50845/50845 [==============================] - 358s 7ms/step - loss: 0.0646 - accuracy: 0.9763 - val_loss: 0.0899 - val_accuracy: 0.9669\n"
     ]
    }
   ],
   "source": [
    "epochs = 3\n",
    "batch_size = 64\n",
    "history = model.fit(X_train, Y_train, epochs=epochs, batch_size=batch_size, validation_split=0.1,\n",
    "                    callbacks=[EarlyStopping(monitor='loss', patience=3, min_delta=0.0001)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6278/6278 [==============================] - 38s 6ms/step\n",
      "Test set\n",
      "  Loss: 0.089\n",
      "  Accuracy: 0.967\n"
     ]
    }
   ],
   "source": [
    "accr = model.evaluate(X_test,Y_test)\n",
    "print('Test set\\n  Loss: {:0.3f}\\n  Accuracy: {:0.3f}'.format(accr[0],accr[1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/matplotlib/font_manager.py:1320: UserWarning: findfont: Font family ['sans-serif'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VOW5wPHfk5kkk50khH1J2GTfMgniVpeqqC3YuqGCrKLtteutrd1sr7e92nqv9XJrqwiIiOvF2msrrUvV2kVJwiL7EiBAQCVkIwEmZJL3/nEmMAkJmZCZOZPM8/185pOZs80zh+E5Z973Pc8RYwxKKaWiQ4zdASillAofTfpKKRVFNOkrpVQU0aSvlFJRRJO+UkpFEU36SikVRTTpK6VUFNGkr6KaiJSIyOftjkOpcNGkr5RSUUSTvlKtEJG7RaRYRCpE5HUR6eebLiLyKxE5IiLHRGSziIz1zbteRLaJSI2IHBKR79j7KZQ6myZ9pVoQkSuBh4Fbgb7AfuAl3+xrgMuAEUCab5ly37xlwD3GmBRgLPBuGMNWKiBOuwNQKgLdCSw3xqwHEJHvA5Uikg3UAynASKDAGLPdb716YLSIfGyMqQQqwxq1UgHQM32lztYP6+weAGNMLdbZfH9jzLvAr4EngCMiskREUn2L3gRcD+wXkb+KyNQwx61UuzTpK3W2w8DgphcikgRkAocAjDGLjTG5wGisZp77fdMLjTEzgF7A74FXwhy3Uu3SpK8UxIqIq+kBvAjME5GJIhIP/Aew1hhTIiJ5IjJFRGKB44AHaBSROBG5U0TSjDH1wDGg0bZPpFQbNOkrBWuAk36Py4EfA68CnwBDgZm+ZVOBp7Ha6/djNfs86ps3GygRkWPAvVh9A0pFFNGbqCilVPTQM32llIoimvSVUiqKaNJXSqkooklfKaWiSMRdkduzZ0+TnZ1tdxhKKdWlrFu37qgxJqu95SIu6WdnZ1NUVGR3GEop1aWIyP72l9LmHaWUiiqa9JVSKopo0ldKqSgScW36Sil1Purr6yktLcXj8dgdSki5XC4GDBhAbGzsea2vSV8p1S2UlpaSkpJCdnY2ImJ3OCFhjKG8vJzS0lJycnLOaxvavKOU6hY8Hg+ZmZndNuEDiAiZmZmd+jWjSV8p1W1054TfpLOfsdskfW9DIw+v2c6hqpN2h6KUUhGr2yT90sqTvFBwgFlL11JWU2d3OEqpKFNVVcVvfvObDq93/fXXU1VVFYKIWtdtkn52zyRWzMvj02oPdy0voPpEvd0hKaWiSFtJ3+v1nnO9NWvW0KNHj1CFdZZuk/QBcgdnsOSuXPYcqWXeigKO1517ZyulVLA88MAD7Nmzh4kTJ5KXl8ell17K9OnTGT16NAA33ngjubm5jBkzhiVLlpxeLzs7m6NHj1JSUsKoUaO4++67GTNmDNdccw0nTwa/ubrbDdm8dHgWi2+fxFefX8c9z61j2Vw38U6H3WEppcLo3/6wlW2HjwV1m6P7pfKTL45pc/4jjzzCli1b2LhxI++//z433HADW7ZsOT20cvny5WRkZHDy5Eny8vK46aabyMzMbLaN3bt38+KLL/L0009z66238uqrrzJr1qygfo5udabfZNrYPvzy5gn8vfgoX39xA94GvT+1Uiq88vPzm42lX7x4MRMmTODCCy/k4MGD7N69+6x1cnJymDhxIgC5ubmUlJQEPa5ud6bf5ObcAdR66vnpH7bx3dWb+M9bJhAT0/2HcymlOOcZebgkJSWdfv7+++/zzjvv8OGHH5KYmMjll1/e6lj7+Pj4088dDoc273TU3ItzqPF4+a+3d5HicvLT6WOiYhyvUir8UlJSqKmpaXVedXU16enpJCYmsmPHDj766KMwR3dGt076APddOYyaOi9LPthLiiuW71x7gd0hKaW6oczMTC6++GLGjh1LQkICvXv3Pj1v2rRpPPnkk4waNYoLLriACy+80LY4xRhj25u3xu12m2DfRMUYww9e28yLBQf5/nUjuedzQ4O6faWU/bZv386oUaPsDiMsWvusIrLOGONub91uf6YP1mXLP7txHDUeLw//aQcprljumDLI7rCUUirsoiLpAzhihF/dNpETpxr44e83k+xyMn1CP7vDUkqpsOqWQzbbEuuI4Td3TiY/O4Nvv7yRd3d8ZndISikVVgElfRGZJiI7RaRYRB5oZf5lIrJeRLwicnOLeb8Uka0isl1EFovNw2dcsQ6WznEzul8qX1m1ng/3lNsZjlJKhVW7SV9EHMATwHXAaOB2ERndYrEDwFzghRbrXgRcDIwHxgJ5wOc6HXUnpbhieXZePoMyEln4bCEfHwxfsSOllLJTIGf6+UCxMWavMeYU8BIww38BY0yJMWYT0PLSVwO4gDggHogFIqJNJT0pjlULp5CZHM+cZwrY+Wnr42uVUqo7CSTp9wcO+r0u9U1rlzHmQ+A94BPf401jzPaWy4nIIhEpEpGisrKyQDYdFL1TXTy/cArxzhhmL1vL/vLjYXtvpVT3cr6llQEef/xxTpw4EeSIWhfSjlwRGQaMAgZgHSiuFJFLWy5njFlijHEbY9xZWVmhDOksAzMSWbVgCvUNjdy5dC2fVnfvmyorpUKjOyX9Q8BAv9cDfNMC8SXgI2NMrTGmFvgTMLVjIYbe8N4pPDs/n6oT9cxatpbyWr0Ji1KqY/xLK99///08+uij5OXlMX78eH7yk58AcPz4cW644QYmTJjA2LFjefnll1m8eDGHDx/miiuu4Iorrgh5nIGM0y8EhotIDlaynwncEeD2DwB3i8jDgGB14j5+PoGG2vgBPVg2x81dywuY80wBL9x9IamuWLvDUkqdjz89AJ9uDu42+4yD6x5pc7Z/aeW33nqL1atXU1BQgDGG6dOn88EHH1BWVka/fv144403AKsmT1paGo899hjvvfcePXv2DG7MrWj3TN8Y4wXuA94EtgOvGGO2ishDIjIdQETyRKQUuAV4SkS2+lZfDewBNgMfAx8bY/4Qgs8RFFOGZPLkrFx2fFLDwhVFnDzVYHdISqku6K233uKtt95i0qRJTJ48mR07drB7927GjRvH22+/zfe+9z3+9re/kZaWFvbYAroi1xizBljTYtqDfs8LsZp9Wq7XANzTyRjD6oqRvXh85kS+9uIGvvL8OpbMdhPnjKpr2JTq+s5xRh4Oxhi+//3vc889Z6e/9evXs2bNGn70ox9x1VVX8eCDD7ayhdDRbNaKL4zvx8NfGsf7O8v41ssbaWiMrKJ0SqnI419a+dprr2X58uXU1tYCcOjQIY4cOcLhw4dJTExk1qxZ3H///axfv/6sdUMtamrvdNTM/EHU1nn52RvbSY538shN47QWv1KqTf6lla+77jruuOMOpk61xq0kJyezatUqiouLuf/++4mJiSE2Npbf/va3ACxatIhp06bRr18/3nvvvZDGGRWllTvjsbd2svjdYhZcksOPbhiliV+pCKWllbW0clB86+oRHPN4Wfb3faS6YvnG54fbHZJSSp03TfrtEBEe/MJoauu8/Ood67aL8y/JaX9FpZSKQJr0AxATIzzy5XEcr/Py0B+3kexycqt7YPsrKqXCyhjT7ZtgO9skr6N3AuR0xPD4zIlcOrwnD7y6iTWbP7E7JKWUH5fLRXl5eaeTYiQzxlBeXo7L5TrvbeiZfgfEOx08NTuXu5YV8I2XNpAU7+RzI8JbK0gp1boBAwZQWlpKOIs22sHlcjFgwFmXRQVMR++ch+qT9dy+5CP2Hq3luQVTyMvOsDskpVSUC3T0jjbvnIe0hFhWLsinX48E5j9TyJZD1XaHpJRSAdGkf556JsezasEUUhNiuWt5AcVHau0OSSml2qVJvxP69Uhg1cIpxIgwa+laDlaEpx62UkqdL036nZTTM4nnFuRz4pSXWcvWcuSY3oRFKRW5NOkHwai+qayYn09ZTR2zlxVQdeKU3SEppVSrNOkHyeRB6Sy9y82+8uPMeaaQ2jqv3SEppdRZNOkH0UXDevLr2yex5VA1i1YW4anXm7AopSKLJv0gu2ZMH/7rlgl8uLec+15YT31Do90hKaXUaZr0Q+DGSf15aPoY3tl+hO/878c06k1YlFIRQsswhMjsqdnU1Hn55Z93khzv5Gc3ju32haCUUpFPk34IffXyYdR4vPz2/T2kJsTyvWkj7Q5JKRXlNOmH2HevvYAaTz2/fX8PKS4nX718mN0hKaWimCb9EBMRHpo+lhqP1dSTEu9k9tRsu8NSSkWpgDpyRWSaiOwUkWIReaCV+ZeJyHoR8YrIzS3mDRKRt0Rku4hsE5Hs4ITedcTECP95ywQ+P6o3P/6/rby2odTukJRSUardpC8iDuAJ4DpgNHC7iIxusdgBYC7wQiubWAk8aowZBeQDRzoTcFcV64jh13dMYuqQTL7zv5t4a+undoeklIpCgZzp5wPFxpi9xphTwEvADP8FjDElxphNQLNB6b6Dg9MY87ZvuVpjTNRWJXPFOnh6jptx/dO474UN/KP4qN0hKaWiTCBJvz9w0O91qW9aIEYAVSLyOxHZICKP+n45RK3keCcr5uUxJCuJu1cWsf5Apd0hKaWiSKgvznIClwLfAfKAIVjNQM2IyCIRKRKRou5+qzOAHolxrFyQT1ZKPHOXF7D9k2N2h6SUihKBJP1DwEC/1wN80wJRCmz0NQ15gd8Dk1suZIxZYoxxG2PcWVnRcc/ZXikuVi2YQlK8k9nLCth39LjdISmlokAgSb8QGC4iOSISB8wEXg9w+4VADxFpyuRXAts6Hmb3NDAjkecWTKHRGGYtXcvhqpN2h6SU6ubaTfq+M/T7gDeB7cArxpitIvKQiEwHEJE8ESkFbgGeEpGtvnUbsJp2/iIimwEBng7NR+mahvVKZuX8fI6drGfWsrUcra2zOySlVDcmxkRWMTC3222KiorsDiPsCksqmL1sLUN6JvPiogtJS4i1OySlVBciIuuMMe72ltMqmxEiLzuDp2a72X2khvkrCjlxSm/CopQKPk36EeRzI7JYPHMSGw5Ucs9z66jz6k1YlFLBpUk/wlw3ri+P3DSev+0+yjde3IhXb8KilAoiTfoR6Fb3QB78wmj+vPVTHvjdZr0Ji1IqaLTKZoSaf0kONR4vv3pnF8nxTn7yxdF6ExalVKdp0o9gX79qGDWeepb+fR+pLiffvuYCu0NSSnVxmvQjmIjwwxtGUePxsvjdYlJcsdx92RC7w1JKdWGa9COciPAfXx5HbZ2Xn6/ZTorLycz8QXaHpZTqojTpdwGOGOFXt03k+Ckv339tM0nxTr44oZ/dYSmluiAdvdNFxDlj+O2dueQNzuBbL2/kvR1ReS8apVQnadLvQhLiHCyb62ZU31TuXbWOj/aW2x2SUqqL0aTfxaS4Ynl2fj4DMxJZ+GwRm0qr7A5JKdWFaNLvgjKS4li1YAo9EmOZs7yA3Z/V2B2SUqqL0KTfRfVJc/H8winEOmK4c+laDpRH7a2HlVIdoEm/CxucmcSqhVM41dDIrGVr+eyYx+6QlFIRTpN+FzeidwrPzsunvLaOWUvXUnH8lN0hKaUimCb9bmDCwB4snZPHgYoTzH2mgBpPvd0hKaUilCb9bmLq0Ex+O2sy2w4fY8GzRXjqtRa/UupsmvS7kStH9uax2yZSWFLBV1at45RXa/ErpZrTpN/NTJ/Qj5/fOI73dpbx7Vc20qC1+JVSfrT2Tjd0x5RB1NbV8x9rdpAc7+ThL4/TWvxKKUCTfre16LKhHDvp5dfvFZPicvKD60dp4ldKadLvzv71mhHUeOp5+m/7SHXF8rWrhtsdklLKZgG16YvINBHZKSLFIvJAK/MvE5H1IuIVkZtbmZ8qIqUi8utgBK0CIyL85Itj+PLk/vzX27tY8Y99doeklLJZu2f6IuIAngCuBkqBQhF53RizzW+xA8Bc4DttbObfgQ86F6o6HzExwi9vGs/xOi8//cM2kl2x3Jw7wO6wlFI2CeRMPx8oNsbsNcacAl4CZvgvYIwpMcZsAs4aIygiuUBv4K0gxKvOg9MRw+LbJ3Hp8J58d/XH/HnLJ3aHpJSySSBJvz9w0O91qW9au0QkBvgv2v4F0LTcIhEpEpGisrKyQDatOije6eCp2blMHNiDr724gQ926X5WKhqFepz+V4E1xpjScy1kjFlijHEbY9xZWVkhDil6JcY5eWZePsN6pXDPc+soKqmwOySlVJgFkvQPAQP9Xg/wTQvEVOA+ESkB/hO4S0Qe6VCEKqjSEmJZOT+fvmku5q0oZOvhartDUkqFUSBJvxAYLiI5IhIHzAReD2Tjxpg7jTGDjDHZWE08K40xZ43+UeGVlRLPcwunkBLv5K5lBewpq7U7JKVUmLSb9I0xXuA+4E1gO/CKMWariDwkItMBRCRPREqBW4CnRGRrKINWnde/RwKrFk5BBGYvXcuhqpN2h6SUCgMxJrJqs7jdblNUVGR3GFFj2+FjzFzyIZnJ8bxyz1SyUuLtDkkpdR5EZJ0xxt3eclpwLcqN7pfKM/Py+bTaw+xla6k+obX4lerONOkrcgen8/RdbvaWHWfuigKO13ntDkkpFSKa9BUAlwzvyeLbJ7GptJpFz+lNWJTqrjTpq9Omje3DL28azz+Ky/naixvwNuhNWJTqbjTpq2Zuyh3AQzPG8Pa2z/ju6k006k1YlOpWtLSyOstdU7Op8Xh59M2dJMU7eWjGGK3Fr1Q3oUlfteqrlw/l2Ml6nvpgL6kJTu6/dqTdISmlgkCTvmqViPDAdSM55vHyxHt7SHHFcu/nhtodllKqkzTpqzaJCD+7cSy1dV4e+dMOUlxO7pwy2O6wlFKdoElfnZMjRnjs1gmcqPPyo99vITneyYyJAVXWVkpFIB29o9oV64jhiTsnMyUng2+/8jHvbPvM7pCUUudJk74KiCvWwdI5eYztl8pXX1jPP/cctTskpdR50KSvApYc72TFvHyyMxO5+9kiNhyotDskpVQHadJXHZKeFMeqBVPITI5n7jOF7Py0xu6QlFIdoElfdVivVBfPL5xCQqyDWcvWUnL0uN0hKaUCpElfnZeBGYmsWpiPt6GRO5eu5ZNqvQmLUl2BJn113ob1SmHl/CkcO1nPrKVrKa+tszskpVQ7NOmrThk3II1lc/MorTzJXcsLOObRm7AoFck06atOy8/J4MnZuez6rIYFKwo5eUpr8SsVqTTpq6C44oJePH7bJNbtr+TeVes45dVa/EpFIk36KmhuGN+XR748nr/uKuObL+tNWJSKRFp7RwXVrXkDOeap52dvbCcpbjO/uGk8MTFai1+pSKFJXwXdwkuHUOPx8t9/2U2KK5Yff2GU3oRFqQgRUPOOiEwTkZ0iUiwiD7Qy/zIRWS8iXhG52W/6RBH5UES2isgmEbktmMGryPXNzw9n3sXZLP/HPv77L7vtDkcp5dPumb6IOIAngKuBUqBQRF43xmzzW+wAMBf4TovVTwB3GWN2i0g/YJ2IvGmMqQpK9CpiiQg/vmE0tR4vj79jnfEvuCTH7rCUinqBNO/kA8XGmL0AIvISMAM4nfSNMSW+ec167owxu/yeHxaRI0AWoEk/CsTECI/cNJ7jp7z8+x+3kRLv5Na8gXaHpVRUC6R5pz9w0O91qW9ah4hIPhAH7Gll3iIRKRKRorKyso5uWkUwR4zwq9smctmILB743Sbe2PSJ3SEpFdXCMmRTRPoCzwHzjDFnjeMzxiwxxriNMe6srKxwhKTCKN7p4KlZueQOTuebL2/g/Z1H7A5JqagVSNI/BPj/Jh/gmxYQEUkF3gB+aIz5qGPhqe4iIc7Bsrl5jOidwr2r1lGwr8LukJSKSoEk/UJguIjkiEgcMBN4PZCN+5Z/DVhpjFl9/mGq7iDVFcvK+fn075HAghWFbDlUbXdISkWddpO+McYL3Ae8CWwHXjHGbBWRh0RkOoCI5IlIKXAL8JSIbPWtfitwGTBXRDb6HhND8klUl5CZHM+qhVNITYjlruUFFB/Rm7AoFU5ijLE7hmbcbrcpKiqyOwwVYiVHj3PLUx8SI7D63osYmJFod0hKdWkiss4Y425vOa29o2yR3TOJ5xbk46lvZNaytRw55rE7JKWigiZ9ZZuRfVJZMS+Pspo6Zi1bS+XxU3aHpFS3p0lf2WrSoHSWznFTUn6CuSsKqa3z2h2SUt2aJn1lu4uG9uSJOyaz5VA1C58txFOvN2FRKlQ06auIcPXo3jx26wTW7qvgX55fT73W4lcqJDTpq4gxY2J//n3GWP6y4wj/+srHNDRG1sgypboDraevIsqsCwdT4/Hyiz/vINnl5Oc3jtVa/EoFkSZ9FXG+cvlQajz1/Ob9PaS4nDwwbaQmfqWCRJO+ikj3X3sBNR4vT/11L6muWP7limF2h6RUt6BJX0UkEeHfpo+hts7Lo2/uJMXl5K6p2XaHpVSXp0lfRayYGOHRm8dTW+flwf/bSnK8ky9PHmB3WEp1aTp6R0U0pyOG/7l9EhcPy+T+1Zt4c+undoekVJemSV9FPFesgyWz3Yzrn8bXXtjA33cftTskpbosTfqqS0iKd7JiXh5DspK4e2UR6/ZX2h2SUl2SJn3VZfRIjGPlgnx6p8Yz75kCth0+ZndISnU5mvRVl9IrxcWqhVNIindy1/K17C2rtTskpboUTfqqyxmQnsiqhVMwBmYtXcuhqpN2h6RUl6FJX3VJQ7OSeXZ+PjV1XmYvXUtZTZ3dISnVJWjSV13W2P5pPDM3j0+qPdy1vIDqE/V2h6RUxNOkr7o0d3YGT83OZc+RWuatKODEKb0Ji1LnoklfdXmXjchi8e0T2XiwinueW0edV2/ColRbuk/Sb6iH1fPh/V/A1t/DkR3g1XuuRotpY/vyy5sn8LfdR/n6ixvw6k1YlGpV96m9c7wMDq2DLb8DfDffiHFCxlDIugCyRkKvkdbfzGHgjLc1XBV8N+cOoMZTz7/9YRvffXUT/3nzBGJitCSzUv4CSvoiMg34b8ABLDXGPNJi/mXA48B4YKYxZrXfvDnAj3wvf2aMeTYYgZ8ltR9842M4dQLKd0PZTijbYZ3xf7YVdvwRjO/sT2IgY4h1AGg6IGRdAD1HQGxCSMJT4THv4hxqPF4ee3sXKfFOfjp9jNbiV8pPu0lfRBzAE8DVQClQKCKvG2O2+S12AJgLfKfFuhnATwA31un3Ot+6obuGPi4R+k6wHv7qPVBebB0Img4IZTtg55/ANLUBC6Rnt34wiE8OWcgquL525TBqPPU8/bd9pCbE8q/XXGB3SEpFjEDO9POBYmPMXgAReQmYAZxO+saYEt+8lg2p1wJvG2MqfPPfBqYBL3Y68o6KdUGfsdbDn/cUVOxpcTDYCcXvQKPfEMAeg1ocDEZaBwNXang/h2qXiPCD60dR4/HyP+8Wk+JysuiyoXaHpVRECCTp9wcO+r0uBaYEuP3W1u3fciERWQQsAhg0aFCAmw4SZxz0GmU9/DXUQ8W+sw8Ge/8KDX4XAqX2P3MQOH1AGAEJ6eH9HKoZEeHnXxpHbZ2X/1izgxRXLLfnh/m7pVQEioiOXGPMEmAJgNvtNjaHY3HEWsk7a0Tz6Y0NUFlypnmo6YBQtBy8fuUAkvuc3YGcNRISM8L6MaKZI0Z47NaJHK/z8oPXNpMU72T6hH52h6WUrQJJ+oeAgX6vB/imBeIQcHmLdd8PcN3IFOOAzKHWY+QNZ6Y3NkL1geYdyGU7YMMqqD9+ZrmkrLP7DLJGQVJP0A7HoItzxvCbO3OZ80wB3355I8nxDq4c2dvusJSyjRhz7hNrEXECu4CrsJJ4IXCHMWZrK8uuAP7YNHrH15G7DpjsW2Q9kNvUxt8at9ttioqKOv5JIlVjIxw71LzzuOl5nV9p4ISMVg4GIyGljx4MgqDGU8+dS9ey89MaVszLZ+rQTLtDUiqoRGSdMcbd7nLtJX3fxq7HGpLpAJYbY34uIg8BRcaY10UkD3gNSAc8wKfGmDG+decDP/Bt6ufGmGfO9V7dLum3xRio+eTsPoMj28FTdWY5V1rrB4PU/now6KCK46e47akPOVx1khfuvpAJA3vYHZJSQRPUpB9OUZP022IM1B45+2BQtgNO+N0mMC7F1+cwqvkBIW0gxHSfC62D7bNjHm5+8p/UeLy8cs9URvROsTskpYJCk353dPyo7wCwvfkBofazM8vEJlpDSZsOAr18B4Ueg63+CMWB8hPc/OQ/AVh970UMyky0OSKlOk+TfjQ5UQFHdzXvQC7bCTWHzyzjdEHP4S2aikZZF6M5ImIQV1jt+qyG2576kKR4J6vvvYg+aS67Q1KqUzTpK/BUQ9muszuQq/0unXDEQebws/sMModaw1a7sU2lVdzx9Fr6pLl45Z6pZCTF2R2SUudNk75qW12N75dBiz6Dyv00K1aXOayVg0H3Klb30d5y5iwvYETvFJ6/ewqpru59oFPdlyZ91XGnjsPR3a0cDPb5FatzQEaO31XITfWJhnfZYnXv7TjC3SuLmDwonWfn55MQp30fquvRpK+Cp1mxOr+movI9rRer6+V/MBgBcUl2Rh+QP3x8mK+/tIHPjchiyWw3cU4dAaW6lkCTfvT14KmOC6RYnX8HcrvF6kZZw03jI2e45Bcn9KO2zsv3f7eZb72ykcUzJ+HQWvyqG9Kkr86ff7G6MX7TzypW5xtiuvd9aPC7m1nqgLP7DLIugAR7Lpq6PX8QtR4vP1+znZR4Jw9/eZzW4lfdjiZ9FXxtFatr8ELV/vaL1aX0beVgEJ5idXdfNoRjnnr+591ikuOd/PCGUZr4VbeiSV+Fj8PZRrG6Bqg6cHYH8vrn2ihW1+JgEORidd++egQ1Hi9L/27dhOXrVw0P2rZVF2QMeOusE5N6zzn+eqD+5Hn+9W2nzziY/VpIP44mfWW/GN+IoIwcuGDamemni9W16EDe9HLrxep6tTggJPc+r4OBiPDgF0afvu1icryT+ZfkBOGDqqBobOhEcm0n6baVzDnPAS/isEa1OV1+f13gTLD+uno0f50R+pv9aNJXkSsmBnoMtB7Drz4z3b9YnX8H8pZXrQvSmpxVrM73SO3X7sEgJkb4xU3jOF7n5aE/biPF5eQW98BzrhMhNhatAAAPg0lEQVSVjLH6aYKaZNs5k/YfJNBRjvjmSdb/ryvNug9Gs+muVpJ2B/5G4AWOOmRTdR+tFqvbYVUuPelXzTsu5ew+g14jrY7lFsXq6rwNLHy2iH8UH+WJOyZz3bi+Yf5QHdTYeB7JtBPNFl7PmWs4OkpiWk++wUi2rf11urp1MUIdp6+Uv+NHz+5APrIDjh85s0xskq8DuvloohOJ/Zn9zDo2lVbxzc+P4KKhmYzplxbYWP6G+uA0QQR65uw/OqqjHHHnmXTbOHNu768jVsuDB5EmfaUCcaLi7A7ks4rVJdCQOYx/VvfkYK0QL6dIiqmnl8vQ09VIWmwDyTFeHA2tnBWfvnitg8JyFhzffJtahbVL04uzlApEYgYMnmo9/Hmqmx0EHGU7uNSzkwZHHXUmjtrGWKq9Dj6rdFBi4qgjjriEHqQmp5CekUpWRg+SEpP1LFhFHE36SrXGlQYD862HHweQ6Hv0AmrrvGw8UMWWkgqK9lewfn8VJw9bZ/cDMxLIy86wHoPTGZqVrGP+le006SvVCcnxTi4Z3pNLhvcEoL6hke2fHKOwpJKikgo+2FXG79YfAiA9MZbcwRnkZaeTl5PB2ED7BZQKIm3TVyqEjDGUlJ+gsKSCopIKikoq2XvUuuAs3hnDxIE9yMvOwJ2dzuTB6VraWZ037chVKkIdra2jqKTy9IFgy+FjNDQaRGBkn1Trl4CvWUjv6KUCpUlfqS7ixCmrX6CwpNLXL1DJ8VNWv8CA9ITTvwTysjMYlpVMjFb/VK3Q0TtKdRGJcU4uGtaTi4ZZ/QLehka2f1Jj/RLYX8Hfi4/y2garXyAtIRb3YKtPIC87nbH904h36lBLFbiAzvRFZBrw31iDF5YaYx5pMT8eWAnkAuXAbcaYEhGJBZYCk7EOMCuNMQ+f6730TF+p5owxHKg4cbpzuLCkgj1lVr9AnDOGiQN6nP4lMHlwOmkJ2i8QjYLWvCMiDmAXcDVQChQCtxtjtvkt81VgvDHmXhGZCXzJGHObiNwBTDfGzBSRRGAbcLkxpqSt99Okr1T7ymvrKNrfdBCoZMuhary+foELeqc0axLq16Nr3sZSdUwwm3fygWJjzF7fhl8CZmAl8CYzgJ/6nq8Gfi3WgGQDJImIE0gATgF+5RGVUucjMzmea8f04doxfQA4eaqBjQerrIPA/kpe23CI5z7aD0D/Hgm4s9NxZ2eQn53B8F7aLxDNAkn6/YGDfq9LgSltLWOM8YpINZCJdQCYAXyCdT3Lt4wxFSilgiohzsHUoZlMHZoJWP0COz6tOX0Q+HBPOf+30Sotkepy4vb7JTCufxquWO0XiBah7sjNBxqAfkA68DcReafpV0MTEVkELAIYNGhQiENSqvtzOmIY2z+Nsf3TmHtxDsYYSitPUrDP6hwuLKnk3R1Wsbk4RwzjB6RZvwRy0skdlEFaovYLdFeBJP1DgH8h8QG+aa0tU+pryknD6tC9A/izMaYeOCIi/wDcQLOkb4xZAiwBq03/PD6HUuocRISBGYkMzEjkptwBAFQcP8W6/Wc6h5f9fS9P/tX673dB75TTvwTc2ekMSE+0M3wVRIEk/UJguIjkYCX3mVjJ3N/rwBzgQ+Bm4F1jjBGRA8CVwHMikgRcCDwerOCVUucvIymOq0f35urRvQHw1Pv1C5RU8vrGwzy/9gAA/dJcuLOtYaLu7AxG9E7Bof0CXVK7Sd/XRn8f8CbWkM3lxpitIvIQUGSMeR1YhpXYi4EKrAMDwBPAMyKyFRDgGWPMplB8EKVU57hiHVw4JJMLh1j9Ag2Nhp2f1pxuDirYV8HrH1v9AikuJ7mDfb8EBqczYWAP7RfoIvSKXKVUQJr6BZoOAkUlFez6rBaw+gXGDUizmoQGZ5A7OJ30pDibI44uWoZBKRVyVSesfoGmg8Cm0mpONVi3TxzeK/l0k1BedgYD0hO0tHQIadJXSoWdp76BTaXVZ6qK7q+kxuMFoE+qq1nn8Mg+qdovEERae0cpFXauWAf5ORnk52QA0Nho2HWkhsKSSgr3WaOE/rjpE8C6F8Hkwenk+WoJTdR+gbDQM32lVFgdqjp5ephoUUklOz+rwRiIdQhj+6ed7hx2Z2eQof0CAdPmHaVUl1B9op71Byop8DUJfXzwTL/A0Kwk8nMycA+27i8wMEP7BdqiSV8p1SV56hvYcqj6dOdw0f5Kqk/WA9ArJb5ZMbmRfVJwOvSWk6Bt+kqpLsoV6/DVBsoAhtLYaCguq7VKSPguHHtj85l+gUmDztxycuLAHiTGaVo7Fz3TV0p1OYerTp4uLV2wr+J0v4AzRhjTP408X5+AOzudnsnxdocbFtq8o5SKGtUnrX6Bpl8CGw9Wccpr9QsMyUoib/CZJqHBmYndsl9Ak75SKmrVeRvYcujYmVFC+yupOmH1C/RMjm928/lRfbtHv4AmfaWU8mlsNOwpqz1zy8n9FRysOAlAYpyDyYPST/8SmDiwB0nxXa9fQJO+Ukqdw6fVHquO0D6rSWj7p8cwBhwxwph+qb5fAunkDs4gKyXy+wU06SulVAcc89Sz4UDV6SahDQeqqPP1C+T0TMI9+EwJiZyeSRHXL6BJXymlOuGUt5Gth606Qk3NQpWn+wXicPt1Do/ul0qszf0CmvSVUiqIjDHsKTt+eoRQYUkFBypOAJAQ62DSoB6nq4pOHpQe9n4BTfpKKRVinx3zUOQ7ABTtr2Db4WM0+voFRvdNbVZVtFeKK6SxaNJXSqkwq63zsuFA5emqohsOVuKpt/oFBmcm4h5s3XzenZ3BkCD3C2gZBqWUCrPkeCeXDs/i0uFZANQ3NLL18JnrBd7feYRX15cC1j2K/TuHx/RLI84Z+n4BPdNXSqkwMcaw7+hxikrOVBUtKbf6BVyxMXx+VG9+fcfk89q2nukrpVSEERGGZCUzJCuZW/MGAnCkxsO6EqtJKCEu9Gf6mvSVUspGvVJcXDeuL9eN6xuW9+v6BSeUUkoFTJO+UkpFkYCSvohME5GdIlIsIg+0Mj9eRF72zV8rItl+88aLyIcislVENotIaAerKqWUalO7SV9EHMATwHXAaOB2ERndYrEFQKUxZhjwK+AXvnWdwCrgXmPMGOByoD5o0SullOqQQM7084FiY8xeY8wp4CVgRotlZgDP+p6vBq4S66qDa4BNxpiPAYwx5caYhuCErpRSqqMCSfr9gYN+r0t901pdxhjjBaqBTGAEYETkTRFZLyLfbe0NRGSRiBSJSFFZWVlHP4NSSqkAhboj1wlcAtzp+/slEbmq5ULGmCXGGLcxxp2VlRXikJRSKnoFkvQPAQP9Xg/wTWt1GV87fhpQjvWr4ANjzFFjzAlgDXB+l5sppZTqtEAuzioEhotIDlZynwnc0WKZ14E5wIfAzcC7xhgjIm8C3xWRROAU8Dmsjt42rVu37qiI7O/Yx2imJ3C0E+uHisbVMRpXx2hcHdMd4xocyELtJn1jjFdE7gPeBBzAcmPMVhF5CCgyxrwOLAOeE5FioALrwIAxplJEHsM6cBhgjTHmjXber1PtOyJSFEj9iXDTuDpG4+oYjatjojmugMowGGPWYDXN+E970O+5B7iljXVXYQ3bVEopZTO9IlcppaJId0z6S+wOoA0aV8doXB2jcXVM1MYVcfX0lVJKhU53PNNXSinVBk36SikVRbpM0u9kpc/v+6bvFJFrwxzXt0Vkm4hsEpG/iMhgv3kNIrLR93g9zHHNFZEyv/df6Ddvjojs9j3mhDmuX/nFtEtEqvzmhXJ/LReRIyKypY35IiKLfXFvEpHJfvNCub/ai+tOXzybReSfIjLBb16Jb/pGEQnqPUgDiOtyEan2+/d60G/eOb8DIY7rfr+Ytvi+Uxm+eaHcXwNF5D1fLtgqIt9oZZnwfMeMMRH/wLo+YA8wBIgDPgZGt1jmq8CTvuczgZd9z0f7lo8HcnzbcYQxriuARN/zrzTF5Xtda+P+mgv8upV1M4C9vr/pvufp4YqrxfJfw7ouJKT7y7fty7CuFt/SxvzrgT8BAlwIrA31/gowroua3g+rEu5av3klQE+b9tflwB87+x0Idlwtlv0i1oWk4dhffYHJvucpwK5W/k+G5TvWVc70O1PpcwbwkjGmzhizDyj2bS8scRlj3jNWCQqAj7DKWIRaIPurLdcCbxtjKowxlcDbwDSb4rodeDFI731OxpgPsC4sbMsMYKWxfAT0EJG+hHZ/tRuXMeafvveF8H2/AtlfbenMdzPYcYXz+/WJMWa973kNsJ2zC1eG5TvWVZJ+Zyp9BrJuKOPytwDrSN7EJVZ10Y9E5MYgxdSRuG7y/YxcLSJN9ZUiYn/5msFygHf9JodqfwWirdhDub86quX3ywBvicg6EVlkQzxTReRjEfmTiIzxTYuI/SVWaZhpwKt+k8Oyv8Rqep4ErG0xKyzfMb0xepiIyCzAjVV/qMlgY8whERkCvCsim40xe8IU0h+AF40xdSJyD9avpCvD9N6BmAmsNs3vv2Dn/opoInIFVtK/xG/yJb791Qt4W0R2+M6Ew2E91r9XrYhcD/weGB6m9w7EF4F/GGP8fxWEfH+JSDLWgeabxphjwdx2oLrKmX5nKn0Gsm4o40JEPg/8EJhujKlrmm6MOeT7uxd4H+voH5a4jHVDm6ZYlgK5ga4byrj8zKTFT+8Q7q9AtBV7KPdXQERkPNa/4QxjTHnTdL/9dQR4jeA1a7bLGHPMGFPre74GiBWRnkTA/vI51/crJPtLRGKxEv7zxpjftbJIeL5joei0CPYD6xfJXqyf+02dP2NaLPMvNO/IfcX3fAzNO3L3EryO3EDimoTVcTW8xfR0IN73vCewmyB1aAUYV1+/518CPjJnOo32+eJL9z3PCFdcvuVGYnWqSTj2l997ZNN2x+QNNO9kKwj1/gowrkFY/VQXtZieBKT4Pf8nMC2McfVp+vfDSp4HfPsuoO9AqOLyzU/DavdPCtf+8n32lcDj51gmLN+xoO3oUD+werZ3YSXQH/qmPYR19gzgAv7X9x+gABjit+4PfevtBK4Lc1zvAJ8BG32P133TLwI2+770m4EFYY7rYWCr7/3fA0b6rTvftx+LgXnhjMv3+qfAIy3WC/X+ehH4BOsezqVYTSX3Yt3fuek/7RO+uDcD7jDtr/biWgpU+n2/inzTh/j21ce+f+cfhjmu+/y+Xx/hd1Bq7TsQrrh8y8zFGtzhv16o99clWH0Gm/z+ra634zumZRiUUiqKdJU2faWUUkGgSV8ppaKIJn2llIoimvSVUiqKaNJXSqkooklfKaWiiCZ9pZSKIv8PuK9SdyGs2EAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f31f92abcc0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title('Loss')\n",
    "plt.plot(history.history['loss'], label='train')\n",
    "plt.plot(history.history['val_loss'], label='test')\n",
    "plt.legend()\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/matplotlib/font_manager.py:1320: UserWarning: findfont: Font family ['sans-serif'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VPW9//HXJyHskIQkoBAIoMiqAgYQxapVK2pd0Na6gKBQbNX+vLfaVq/VtvZa7a211ta2VxEF16LWalt7xQVub+sCYd/3LQlCgCRsCWT5/v44J8kkApnA7PN+Ph55cOacMzOfHCbv+c5ZPmPOOUREJDmkRLsAERGJHIW+iEgSUeiLiCQRhb6ISBJR6IuIJBGFvohIElHoi4gkEYW+JAwzm2tmpWbWJtq1iMQqhb4kBDPrDZwHOOCqCD5vq0g9l0goKPQlUdwCfAq8AEysm2lm7czsl2a2xczKzeyfZtbOXzbGzD42szIz22Zmk/z5c81sSsBjTDKzfwbcdmZ2p5mtA9b5837tP8ZeM1tgZucFrJ9qZv9hZhvMbJ+/vKeZPW1mvwz8JczsHTP793BsIBFQ6EviuAV42f+51My6+fMfB84CzgG6AN8Has0sD/g78BsgBxgKLG7B810DjAIG+bfn+4/RBXgFeN3M2vrLvgvcCFwOdAZuAw4CM4AbzSwFwMyygYv9+4uEhUJf4p6ZjQHygFnOuQXABuAmP0xvA+52zhU552qccx875w4BNwEfOOdedc5VOed2O+daEvqPOuf2OOcqAJxzL/mPUe2c+yXQBujvrzsF+KFzbo3zLPHXnQeUAxf5690AzHXO7TjBTSJyVAp9SQQTgdnOuV3+7Vf8edlAW7w3gaZ6HmV+sLYF3jCze81slb8LqQxI95+/ueeaAYz3p8cDL55ATSLN0kEoiWv+/vnrgVQz+9yf3QbIAE4GKoFTgCVN7roNGHmUhz0AtA+4fdIR1qlvT+vvv/8+3oh9hXOu1sxKAQt4rlOA5Ud4nJeA5WZ2JjAQ+PNRahIJCY30Jd5dA9Tg7Vsf6v8MBP4Pbz//dOAJM+vuH1Ad7Z/S+TJwsZldb2atzCzLzIb6j7kYuNbM2pvZqcDkZmroBFQDJUArM3sIb999nWnAT82sn3nOMLMsAOdcId7xgBeBN+t2F4mEi0Jf4t1E4Hnn3Fbn3Od1P8BvgZuB+4BleMG6B/g5kOKc24p3YPUef/5i4Ez/MX8FHAZ24O1+ebmZGt4D/gdYC2zB+3QRuPvnCWAWMBvYCzwHtAtYPgM4He3akQgwfYmKSHSZ2ZfwdvPkOf1BSphppC8SRWaWBtwNTFPgSyQo9EWixMwGAmV4B5yfjHI5kiS0e0dEJIlopC8ikkRi7jz97Oxs17t372iXISISVxYsWLDLOZfT3HoxF/q9e/emoKAg2mWIiMQVM9sSzHravSMikkQU+iIiSUShLyKSRGJun/6RVFVVUVhYSGVlZbRLCbu2bduSm5tLWlpatEsRkQQUF6FfWFhIp06d6N27N2bW/B3ilHOO3bt3U1hYSJ8+faJdjogkoLjYvVNZWUlWVlZCBz6AmZGVlZUUn2hEJDriIvSBhA/8Osnye4pIdMTF7h0RkURVXVPL6s/3sWhbGalm3DSqV1ifT6EfpLKyMl555RXuuOOOFt3v8ssv55VXXiEjIyNMlYlIPNm9/xALt5axaGspC7eWsrSwnIOHawAY3itDoR8rysrK+N3vfveF0K+urqZVq6NvxnfffTfcpYlIjKobxS/cWsqirWUs3FrKlt0HAWiVYgzq3pnr83syrFcGw3tlkpvZrplHPHEK/SDdd999bNiwgaFDh5KWlkbbtm3JzMxk9erVrF27lmuuuYZt27ZRWVnJ3XffzdSpU4GGthL79+/nsssuY8yYMXz88cf06NGDt99+m3btwv+fLCKRsWv/IRZuKa0fyS8tLKeiyhvF53Rq443kR/ZieF4mQ7qn0651asRrjLvQ/8lfVrCyeG9IH3NQ98786MrBx1znscceY/ny5SxevJi5c+dyxRVXsHz58vpTK6dPn06XLl2oqKhgxIgRXHfddWRlZTV6jHXr1vHqq6/y7LPPcv311/Pmm28yfvz4kP4uIhIZVTW1rN7ujeLrRvJb9zSM4gd378w3RvRkeF4mw3pmkJvZLiZO1Ii70I8VI0eObHQu/VNPPcVbb70FwLZt21i3bt0XQr9Pnz4MHep99/ZZZ53F5s2bI1aviJyYkn2HGgX80sIyKqtqAejaqQ3De2Uy/uxeDO+VyZAe6bRNi/woPhhxF/rNjcgjpUOHDvXTc+fO5YMPPuCTTz6hffv2XHDBBUc8175Nmzb106mpqVRUVESkVhFpmaqaWlZt31u/q2bh1lIKS72/17RUY1D3dG4c6QX88LxMuqe3jYlRfDDiLvSjpVOnTuzbt++Iy8rLy8nMzKR9+/asXr2aTz/9NMLViciJ2LmvkoVbGp9Rc6jaG8V36+yN4ieO7s3wvAwGd4/dUXwwFPpBysrK4txzz2XIkCG0a9eObt261S8bO3Ysf/jDHxg4cCD9+/fn7LPPjmKlInIsh6v9UfxWfxS/pZSiMm8U3zo1hcE9OnPzqDyG53ln1HTPSKyTLWLuO3Lz8/Nd0y9RWbVqFQMHDoxSRZGXbL+vSDjt2Fvpj+C9gF9W1DCKPzm9LcN7ZTKsVwbDemUyuHvnuB3Fm9kC51x+c+tppC8iCeNwdS0risvrz4lftLWs0Sh+SI/OjD87z98Xn8HJ6Yk1ig+GQl9E4tbn5ZV+uHsj+WVF5Rz2R/Hd09syLC+T28b0YVivDAZ370ybVvE5ig8lhb6IxIVD1TWsKPbOqFm0rYxFW0opLvfOkmvdKoXTe6QzcXQew3plMrxXJielt41yxbFJoS8iMWl7eQULt5TVj+SXF+3lcI03iu+R0Y7heZlM8ffHD9IoPmgKfRGJukPVNSwv2suigB412wNG8Wf0SGfSub0Z7h9w7dZZo/jjpdAXkYgrLqvwTpncUsaibaWsaDKKz+/dheF+E7KBJ3emdau4+eqPmKfQD9LxtlYGePLJJ5k6dSrt27cPQ2Uisa2yqoYVxeUBu2rK+HyvN4pv0yqFM3LTufXc3v6++Ay6ahQfVgr9IB2ttXIwnnzyScaPH6/Ql4TnnKO4vNJvX+CdUbOyuJyqGu96oJ5d2jGyjz+Kz/NG8WmpGsVHkkI/SIGtlS+55BK6du3KrFmzOHToEOPGjeMnP/kJBw4c4Prrr6ewsJCamhoefPBBduzYQXFxMRdeeCHZ2dnMmTMn2r+KSMhUVtWwvKi8flfNwq2l7Nx3CIC2aSmc0SOD28b0qb8AqmsnjeKjLf5C/+/3wefLQvuYJ50Olz12zFUCWyvPnj2bN954g3nz5uGc46qrruIf//gHJSUldO/enb/97W+A15MnPT2dJ554gjlz5pCdnR3aukUiyDlHYWkFi7Z5V7Yu2lrKyu1760fxvbq0Z/QpWd6FT70yGXByJ43iY1D8hX4MmD17NrNnz2bYsGEA7N+/n3Xr1nHeeedxzz338IMf/ICvfvWrnHfeeVGuVOT4VVbVsKyovNGumhJ/FN8uLZUzctOZPKZv/Rk1OZ3aNPOIEgviL/SbGZFHgnOO+++/n9tvv/0LyxYuXMi7777LD3/4Qy666CIeeuihKFQo0jJ1o/jAr/VbWbyX6lpvFJ+X1Z4xp2bXf63fgJM60Uqj+LgUf6EfJYGtlS+99FIefPBBbr75Zjp27EhRURFpaWlUV1fTpUsXxo8fT0ZGBtOmTWt0X+3ekVhRcbiGpYVl9btqFm4tY9f+hlH8mT3Tmfqlvgzz98Vnd9QoPlEEFfpmNhb4NZAKTHPOPdZkeR4wHcgB9gDjnXOFZnYh8KuAVQcANzjn/hyK4iMpsLXyZZddxk033cTo0aMB6NixIy+99BLr16/ne9/7HikpKaSlpfH73/8egKlTpzJ27Fi6d++uA7kScc45tu2paNSjZtX2hlF876z2fKlfNsP8r/XTKD6xNdta2cxSgbXAJUAhMB+40Tm3MmCd14G/OudmmNmXgVudcxOaPE4XYD2Q65w7eLTnU2vl5Pt9JbQOHq5maWF5/a6aRVtL2bX/MADtW6dyZm4Gw/MyGNbTG8VnaRSfEELZWnkksN45t9F/4NeAq4GVAesMAr7rT88BjjSS/xrw92MFvoi0jHOOrXsONrq6ddX2fdT4o/g+2R340mk59adM9u+mUXyyCyb0ewDbAm4XAqOarLMEuBZvF9A4oJOZZTnndgescwPwxJGewMymAlMBevXqFVzlIkno4OFqlmwrr99Vs2hrGbsPeKP4Dq1TObNnBt8+/xSG52UwtGcmXTq0jnLFEmtCdSD3XuC3ZjYJ+AdQBNTULTSzk4HTgfeOdGfn3DPAM+Dt3jnKOnHzxcMnIta+yUyixznHlt3+KN4fya/Z0TCK75vdgQv6d63/Wr/TunUiNSXx/0bkxAQT+kVAz4Dbuf68es65YryRPmbWEbjOOVcWsMr1wFvOuarjKbJt27bs3r2brKyshA5+5xy7d++mbVtdtZiMDhyqZklhmXfKpN8zfo8/iu/YphVn9kznjgtOYXivTIb2zCBTo3g5DsGE/nygn5n1wQv7G4CbAlcws2xgj3OuFrgf70yeQDf6849Lbm4uhYWFlJSUHO9DxI22bduSm5sb7TIkzJxzbN59sNGFT2s+34s/iKdvTge+PKBr/df69euqUbyERrOh75yrNrO78HbNpALTnXMrzOxhoMA59w5wAfComTm83Tt31t3fzHrjfVL43+MtMi0tjT59+hzv3UWibv+hapZuK6sP+EVbSyk96H3w7dSmFUN7ZXDJhafWnzaZ0V6jeAmPZk/ZjLQjnbIpEk+cc2zcdaD+ytaFW0pZu2Nf/Sj+1K4d61sXDO+VyaldO2oULycslKdsisgx7KusYsm2cv/CJ29ffFmTUfxXBp/kBX3PTNLbp0W5YklmCn2RFqit9UbxgRc+rdmxj7oPzP26duTSQSd5PWryMjk1pyMpGsVLDFHoixxN9SHYvR5KVlOzYzXLlxZQWl5ORU0K7UhldGprxnZoT2Zee7p07khOegfatG4Dqa2hIg3WpcGGNEj1f1KONd3an24VxHQapOgCKzk+Cn2RQ/th11rvp2Q1lPj/lm4C531vq5FCem0OWe0607E1tE+pIc1qsNpq2HcYyqqgpgpqq6DmcPhrtlT/TaO1/4ZwtOmWvJkETDd6Ywr2OYJ4g6u7TwKfeh3rFPqSPCpKGwI9MODLtzask9IKsk6Fk4bAkOtwOf353fJUnloC/zb2DL59wSnNP49zUFvjvwEEvhn4bwi11d6/NVVHnz7ifY423cxj1VZDdSUc2tfkcY8yXXtcl9O0TN2bRUoapB5tOphPSMd4Yzni9PE8R+vG90lJjes3LYW+JBbn4EAJlKxpEu5rYP+OhvVatYXsftBrFGTfAjn9IWcAdOnj/bH7npi9ht8sWc/t5/cNLvDBC4TUVt5PWrsQ/4IR4FyQbyYBbxjNvjHVTTfzhnO0xz18IPg3P1fT/O94okLyhtX6i/fP6AUjpoS1dIW+xCfnYG9R490xdQFfUdqwXutOXqCferH3b3Z/79+MXt6I7Rim/d9GfvPRem4Y0ZP7xg4I8y8UQ8wawike1dY2/tRyrDeTRm9G1UeZDuZN6vAx7l8NVRXBPW73YQp9SXK1NVC2pWHkHhjwh/c3rNeuizdSH3SNP2r3A75z9+P6KP56wTb+82+ruPz0k3hk3OkJ3f4j4aSkQEobaKWW0Uei0JfYUH0Y9myEXWv8gPd/dq/z9kfX6XQyZJ8GQ2+GnNO8oM8ZAB1C961k7634nB+8uZTz+mXzq28M1YVTklAU+hJZVRWwa13AvnZ/9L5ng/dxu05GLy/M+57vB3t/L+zbZYS1vI/X7+I7ryzizJ4Z/GH8WbRpdexdQCLxRqEv4XFoX8CumDUNu2dKtwD+lUyW6h04zRkAA64ICPd+0LpDxEtesq2Mb84soE92B56fNIIObfTnIYlHr2o5MQf3NJwdU7KmIeD3BnTfTm3tnQbZfRiceaM3Ys8ZAFmnxMx+13U79jHp+Xl06diamZNHquGZJCyFvjTPOe90x8ADqXUBfyCg3XVaey/Qe49pOAUyuz9k9vZOR4tRhaUHmfDcPFqlpvDS5FF066zvM5DEFbt/iRJ5tbWwtzDgTJmA0XtlecN6bdK9UD9tbEO45/SHzrlx1x6gZN8hJjw3j4OHq/nj7aPJy4r8biWRSFLoJ6Oaaijd7O+KCQz3tVAV8L31HXK8QB/yNT/Y/d0yHbvF9RWJdcorqpg4fR7byyt4ecooBp7cOdoliYSdQj+RVR+C3Ru+eGXq7vWN+8N07uGN1IdPbDxyb98lerWHWcXhGqbMmM+6nfuYNnEEZ+Ul7u8qEkihnwgOH/BOg2zaemDPpoBL0g0y87xA73eJf2XqAO9MmbbJNcKtqqnljpcXULCllN/cOIzzT8uJdkkiEaPQjycVZX6gr2l8MLWsScOwLqdA14EweJwf7Kd54R6PfWBCrLbWcc+sJcxZU8LPxp3OV8/oHu2SRCJKoR+LDuxqfOFS3eh93/aGdVLbeGGeOxKGTWjYLZPZB1rpdMMjcc7xo3dW8M6SYr4/tj83jeoV7ZJEIk6hHy3Owd7ixhcu1QV8xZ6G9Vp39MK974UNPWVy+kNGXrMNw6SxX72/lhc/3cLtX+rLt88PsmOmSIJR6IdbbW1Dw7CmfWUO72tYr12mN1IfeGXDgdSc/t5B1gQ4UybanvvnJp76aD3fyO/JfZcNUAM1SVoK/VCpqfIOnDZtO7BrPVRXNKzXsZsX5kMDrkytaximIAqLNxYU8tO/ruSyISfxs2vVMVOSm0K/paoqvc6PTdsO7N7Q+BuH0nt557X3OT+gj/tp3oheIqauY+aYU7N58gZ1zBRR6B/NoX3+mTJN2g6Ubq7/3lQsxTtwmjMA+l8WcKbMadCmY1TLF/h4g9cx8/Qe6fz3BHXMFAGFvtcwrOkXYpes8doR1ElJ87839Qw4/fqGK1O7nAJp6tMSi5YWlvHNGQX0zm6vjpkiAZLjL8E52L/zi1emlqyBAzsb1mvVzgv0vHMaX5ma2SemG4ZJY+t37mPi9HlkdmjNi5NHkdlBp7CK1EmsJHMOygsD9rUHhHtlWcN6bTr7DcO+0nBlas5p3n74OGsYJo0Vlh5k/LR5pKaoY6bIkSRO6JcXwdMjG39vavtsL9yHXNuwvz1nAHQ6SWfKJKBd+xt3zOydrY6ZIk0lTuh3Osm/MvU0f/TeP6TfmyqxbW9lFbc8p46ZIs1JnNBPSYXLHot2FRIFFYdrmPJCAet27uPZW/LVMVPkGILagW1mY81sjZmtN7P7jrA8z8w+NLOlZjbXzHIDlvUys9lmtsrMVppZ79CVL8muqqaWO19ZyPwte3ji+qFc0L9rtEsSiWnNhr6ZpQJPA5cBg4AbzWxQk9UeB2Y6584AHgYeDVg2E/iFc24gMBLYiUgI1NY67n19CR+t3sl/XjOEK89Ux0yR5gQz0h8JrHfObXTOHQZeA65uss4g4CN/ek7dcv/NoZVz7n0A59x+59xBRE6Qc44f/2UFby8u5nuX9ufmUXnRLkkkLgQT+j2AbQG3C/15gZYA1/rT44BOZpYFnAaUmdmfzGyRmf3C/+TQiJlNNbMCMysoKSlpuljkC371wTpmfrKFqV/qyx0XqGOmSLBCdVL6vcD5ZrYIOB8oAmrwDhSf5y8fAfQFJjW9s3PuGedcvnMuPydH32Ikxzb9n5t46sN1fCO/J/erY6ZIiwQT+kVAz4Dbuf68es65Yufctc65YcAD/rwyvE8Fi/1dQ9XAn4HhIalcktKbCwp5+K8rGTv4JB4ZN0SBL9JCwYT+fKCfmfUxs9bADcA7gSuYWbaZ1T3W/cD0gPtmmFnd8P3LwMoTL1uS0ewVn/P9N5dy7qlZ/PrGobRK1dXTIi3V7F+NP0K/C3gPWAXMcs6tMLOHzewqf7ULgDVmthboBjzi37cGb9fOh2a2DDDg2ZD/FpLwPtmwm7teXcSQHuk8MyFfHTNFjpM556JdQyP5+fmuoKAg2mVIDFlaWMZNz37GyeltmXX7aDVQEzkCM1vgnMtvbj19PpaYVtcxM6N9mjpmioSAQl9iVlFZBROea+iYeVK6OmaKnCiFvsSkXfsPMWHaZ+w/VM3M20aqY6ZIiCj0Jebsraxi4vR5FJdX8PykEQzqro6ZIqGi0JeYUlnldcxc8/k+/jD+LPJ7q2OmSCglTmtliXtVNbXc+bLXMfPXNwxTx0yRMNBIX2JCba3je68v4cPVO/np1UO4Sh0zRcJCoS9R55zjJ39ZwZ/9jpnjz1bHTJFwUehL1D35wTpmfLKFb57XRx0zRcJMoS9RNf2fm/j1h+u4Pj+X/7h8oBqoiYSZQl+ipq5j5qWDu/Gzcacr8EUiQKEvUfH+yh0NHTNvGKaOmSIRor80ibhPNuzmzlcWMqRHOv89IZ+2aeqYKRIpCn2JqGWF5XxzZgF5XdrzwqQRdGyjS0VEIkmhLxGzfud+Jj4/j/R26pgpEi0KfYkIr2PmZ6SY8fIUdcwUiRaFvoSdOmaKxA6FvoRVYMfM6eqYKRJ1Cn0Jm8qqGqbM8Dpm/n78WYxQx0yRqNOpExIWVTW13PXKQuZv3sOT3xjKheqYKRITNNKXkKutdXz/jaV8sGonD189hKuH9oh2SSLiU+hLSDnnePivK3lrURH3fuU0JqhjpkhMUehLSP36w3W88PFmpozpw50XnhrtckSkCYW+hMzz/9rEkx+s4+tn5fLAFeqYKRKLFPoSEn9aWMhP/rKSrwzqxqPXqmOmSKxS6MsJ+2DlDr73xlLOOSWLp25Ux0yRWKa/Tjkhn27czR2vLGRI9848c4s6ZorEOoW+HLflReVMmVFAry7teeHWkeqYKRIHFPpyXNbv3M8t0+s6Zo5Ux0yROKHQlxYrKqvgluc+I8XgpSmjODm9XbRLEpEgKfSlRXbvP8SE5z5jX2U1M24bSR91zBSJK0GFvpmNNbM1ZrbezO47wvI8M/vQzJaa2Vwzyw1YVmNmi/2fd0JZvETWvsoqJj4/j+KyCqbfOoLB3dOjXZKItFCzR97MLBV4GrgEKATmm9k7zrmVAas9Dsx0zs0wsy8DjwIT/GUVzrmhIa5bIqyuY+bq7ft49pZ8dcwUiVPBjPRHAuudcxudc4eB14Crm6wzCPjIn55zhOUSx+o6Zs7bvIdfXn8mFw5Qx0yReBVM6PcAtgXcLvTnBVoCXOtPjwM6mVmWf7utmRWY2admds2RnsDMpvrrFJSUlLSgfAm32lrHD9QxUyRhhOpA7r3A+Wa2CDgfKAJq/GV5zrl84CbgSTM7pemdnXPPOOfynXP5OTk5ISpJTlRdx8w/qWOmSMII5mqaIqBnwO1cf14951wx/kjfzDoC1znnyvxlRf6/G81sLjAM2HDClUvYPfXhel74eDOT1TFTJGEEM9KfD/Qzsz5m1hq4AWh0Fo6ZZZtZ3WPdD0z352eaWZu6dYBzgcADwBKjXvjXJn71wVq+dlYuD1yujpkiiaLZ0HfOVQN3Ae8Bq4BZzrkVZvawmV3lr3YBsMbM1gLdgEf8+QOBAjNbgneA97EmZ/1IDHprUSE/9jtmPnbt6aSkKPBFEoU556JdQyP5+fmuoKAg2mUkrQ9W7uD2lxYwsncXnr91hBqoicQJM1vgHz89Jl2RK/U+3bibO/2Omc9OVMdMkUSk0BegoWNmzy7teV4dM0USlkJf2FCyn4kBHTO7qGOmSMJS6Ce54rIKJkz7DDN4cfJIdcwUSXAK/SQW2DHzhVtH0jenY7RLEpEw047bJLWvsopJz8+nsLSCFyePYkgPdcwUSQYa6SehyqoavjmzgFXb9/L78cMZ2UcdM0WShUb6Saa6ppa7XlnEZ5v28OQ3hvLlAd2iXZKIRJBG+kmkttbx/TeX8sGqHfzkqsHqmCmShBT6ScI5x0//tpI/LSzinktO45bRvaNdkohEgUI/Sfzmo/U8/6/N3HZuH+76sjpmiiQrhX4SmPHxZp54fy3XDc/lh1eoY6ZIMlPoJ7g/LyriR++s4JJB3fj5deqYKZLsFPoJ7MNVO7jn9SWM7pvFb24cRqtU/XeLJDulQIL6bONu7nh5IYPVMVNEAij0E1Bdx8zczHa8oI6ZIhJAoZ9gNvodMzu3S+OlKaPUMVNEGlHoJ5DisgomPDcPUMdMETkyhX6C2HPgMBOe+4y9FVXMuE0dM0XkyLSzNwF4HTPnqWOmiDRLI/04V1lVw9SZC1hZrI6ZItI8jfTjWHVNLd95dRGfbNytjpkiEhSN9ONUba3jB28u4/2VXsfMa4apY6aINE+hH4ecc/zn31bx5sJCvnvJaUw8p3e0SxKROKHQj0O//Wg90/+1iVvP7c131DFTRFpAoR9nZn6ymV++v5Zrh/fgwSsGqWOmiLSIQj+OvL24iIfeXsHFA7vxX9edoY6ZItJiCv048dHqHdwzawln9+3Cb29Sx0wROT5Kjjgwb9Mevv3SQgae3Jlnb1HHTBE5fgr9GLe8qJzJL8z3O2aOoFPbtGiXJCJxLKjQN7OxZrbGzNab2X1HWJ5nZh+a2VIzm2tmuU2WdzazQjP7bagKTwZ1HTM7tW3Fi5NHkdWxTbRLEpE412zom1kq8DRwGTAIuNHMBjVZ7XFgpnPuDOBh4NEmy38K/OPEy00e28sDOmZOGUX3DHXMFJETF8xIfySw3jm30Tl3GHgNuLrJOoOAj/zpOYHLzewsoBsw+8TLTQ57Dhxm/LSGjpmnqGOmiIRIMKHfA9gWcLvQnxdoCXCtPz0O6GRmWWaWAvwSuPdYT2BmU82swMwKSkpKgqs8Qe0/VF3fMXPaxHx1zBSRkArVgdx7gfPNbBFwPlAE1AB3AO865wqPdWfn3DPOuXznXH5OTk6ISoo/lVU1fHNGASuK9/K7m4czqm9WtEsSkQQTTJfNIqBnwO1cf14951wx/kjfzDoC1znnysxsNHCemd0BdARam9l+59wXDgYnu+qaWv5fQMfMiwaqY6a1y7h/AAALvklEQVSIhF4woT8f6GdmffDC/gbgpsAVzCwb2OOcqwXuB6YDOOduDlhnEpCvwP+i2lrHfX9axmx1zBSRMGt2945zrhq4C3gPWAXMcs6tMLOHzewqf7ULgDVmthbvoO0jYao34TjneOTdVbyxoJB/v1gdM0UkvMw5F+0aGsnPz3cFBQXRLiNifvvROh6fvZZbz+3NQ19VAzUROT5mtsA5l9/ceroiN4pe/GQzj89ey7XD1DFTRCJDoR8lby8u4qF3vI6ZP/+aOmaKSGQo9KNgzuqd3DNrCSN7ex0z09QxU0QiRGkTYfM27eFbLy1gwMmdmDZRHTNFJLIU+hG0otjrmNkjsx0zbh2pjpkiEnEK/QjZtOtAfcfMl9QxU0SiRKEfAdvLKxg/7TNqnTpmikh0KfTDbM+Bw0x4bh7lFVXMVMdMEYkyhX4Y7T9Uza3Pz2PbnoPqmCkiMSGY3jtyHCqrapg6s4DlxXv57/FncbY6ZopIDNBIPwzqOmZ+vGE3j3/9DC4epI6ZIhIbFPohFtgx88dXDmLcsNzm7yQiEiEK/RByzvEzv2Pmv13cj0nn9ol2SSIijSj0Q+h3czcw7Z+bmHROb+6+qF+0yxER+QKFfoi8+OkWfvHeGsYN66EWySISsxT6IfD24iIeens5Fw/syn+pY6aIxDCF/glq3DFzuDpmikhMU0KdgPmb9/Dtl9UxU0Tih0L/OK0s3sttL8yne4Y6ZopI/FDoH4dNuw5wy/R5dGrTihfVMVNE4ohCv4U+L6/0O2Y6Xpwyih7qmCkicUSh3wKlBw4z4bnPKK+oYsat6pgpIvFHDdeCtP9QNZNemM+WPQeZedtITs9Vx0wRiT8a6QehvmNmUTlP3zRcHTNFJG4p9JtRXVPL3a95HTN/8bUzuEQdM0Ukjin0j8E5x/1/WsZ7K3bwoysHce1wdcwUkfim0D8K5xyP/G0Vry8o5O6L+nGrOmaKSAJQ6B9FYMfMf7tYHTNFJDEo9I/gJXXMFJEEpdBv4p0lxTz49nIuGqCOmSKSeIIKfTMba2ZrzGy9md13hOV5ZvahmS01s7lmlhswf6GZLTazFWb2rVD/AqE0d81OvvvHxYzo3YWnb1bHTBFJPM2mmpmlAk8DlwGDgBvNbFCT1R4HZjrnzgAeBh71528HRjvnhgKjgPvMrHuoig+lgs17+NZLC+h/kjpmikjiCmYoOxJY75zb6Jw7DLwGXN1knUHAR/70nLrlzrnDzrlD/vw2QT5fxK0s3sutL8yne3o7Ztw2ks7qmCkiCSqYEO4BbAu4XejPC7QEuNafHgd0MrMsADPraWZL/cf4uXOuuOkTmNlUMysws4KSkpKW/g4nZLPfMbNjm1a8OGUU2eqYKSIJLFQj73uB881sEXA+UATUADjntvm7fU4FJprZFy5pdc4945zLd87l5+TkhKik5n1eXsn45/yOmZPVMVNEEl8woV8E9Ay4nevPq+ecK3bOXeucGwY84M8ra7oOsBw474QqDpG6jpllB72Omad2VcdMEUl8wYT+fKCfmfUxs9bADcA7gSuYWbaZ1T3W/cB0f36umbXzpzOBMcCaUBV/vA4EdMx89pZ8dcwUkaTRbOg756qBu4D3gFXALOfcCjN72Myu8le7AFhjZmuBbsAj/vyBwGdmtgT4X+Bx59yyEP8OLXKouoapLzZ0zBx9ijpmikjyMOdctGtoJD8/3xUUFITlsatrarnrlUX8z4rP+eXXz+S6s9RATUQSg5ktcM7lN7deTJ5CGQ7OOf7jrWX8z4rPeeirgxT4IpKUkiL0nXP87N1VzCoo5P9d1I/bxqhjpogkp6QI/d/N3cCz/7eJiaPz+Hd1zBSRJJbwof/yZ17HzGuGdudHVw5Wx0wRSWoJHfp/WVLMD//sdcz8xdfPVMdMEUl6CRv6c9fs5Luz1DFTRCRQQiZhXcfM07qpY6aISKCEC/1V2/dymzpmiogcUUKF/uZdB5jw3Dw6tGnFzMkj1TFTRKSJhAn9HXu9jpk1tbW8OHkkuZnto12SiEjMaRXtAkKlXetU+nfrxN0X9+PUrp2iXY6ISExKmNDv3DaN5yaNiHYZIiIxLWF274iISPMU+iIiSUShLyKSRBT6IiJJRKEvIpJEFPoiIklEoS8ikkQU+iIiSSTmvhjdzEqALSfwENnArhCVE0qqq2VUV8uorpZJxLrynHM5za0Uc6F/osysIJhvhI801dUyqqtlVFfLJHNd2r0jIpJEFPoiIkkkEUP/mWgXcBSqq2VUV8uorpZJ2roSbp++iIgcXSKO9EVE5CgU+iIiSSRuQt/MxprZGjNbb2b3HWF5GzP7o7/8MzPrHbDsfn/+GjO7NMJ1fdfMVprZUjP70MzyApbVmNli/+edCNc1ycxKAp5/SsCyiWa2zv+ZGOG6fhVQ01ozKwtYFs7tNd3MdprZ8qMsNzN7yq97qZkND1gWzu3VXF03+/UsM7OPzezMgGWb/fmLzawgwnVdYGblAf9fDwUsO+ZrIMx1fS+gpuX+a6qLvyyc26unmc3xs2CFmd19hHUi8xpzzsX8D5AKbAD6Aq2BJcCgJuvcAfzBn74B+KM/Pchfvw3Qx3+c1AjWdSHQ3p/+dl1d/u39Udxek4DfHuG+XYCN/r+Z/nRmpOpqsv53gOnh3l7+Y38JGA4sP8ryy4G/AwacDXwW7u0VZF3n1D0fcFldXf7tzUB2lLbXBcBfT/Q1EOq6mqx7JfBRhLbXycBwf7oTsPYIf5MReY3Fy0h/JLDeObfROXcYeA24usk6VwMz/Ok3gIvMzPz5rznnDjnnNgHr/ceLSF3OuTnOuYP+zU+B3BA99wnVdQyXAu875/Y450qB94GxUarrRuDVED33MTnn/gHsOcYqVwMznedTIMPMTia826vZupxzH/vPC5F7fQWzvY7mRF6boa4rkq+v7c65hf70PmAV0KPJahF5jcVL6PcAtgXcLuSLG6x+HedcNVAOZAV533DWFWgy3jt5nbZmVmBmn5rZNSGqqSV1Xed/jHzDzHq28L7hrAt/N1gf4KOA2eHaXsE4Wu3h3F4t1fT15YDZZrbAzKZGoZ7RZrbEzP5uZoP9eTGxvcysPV5wvhkwOyLby7xdz8OAz5osishrLGG+GD3Wmdl4IB84P2B2nnOuyMz6Ah+Z2TLn3IYIlfQX4FXn3CEzux3vU9KXI/TcwbgBeMM5VxMwL5rbK6aZ2YV4oT8mYPYYf3t1Bd43s9X+SDgSFuL9f+03s8uBPwP9IvTcwbgS+JdzLvBTQdi3l5l1xHuj+Tfn3N5QPnaw4mWkXwT0DLid68874jpm1gpIB3YHed9w1oWZXQw8AFzlnDtUN985V+T/uxGYi/fuH5G6nHO7A2qZBpwV7H3DWVeAG2jy0TuM2ysYR6s9nNsrKGZ2Bt7/4dXOud118wO2107gLUK3W7NZzrm9zrn9/vS7QJqZZRMD28t3rNdXWLaXmaXhBf7Lzrk/HWGVyLzGwnHQItQ/eJ9INuJ93K87+DO4yTp30vhA7ix/ejCND+RuJHQHcoOpaxjegat+TeZnAm386WxgHSE6oBVkXScHTI8DPnUNB402+fVl+tNdIlWXv94AvINqFontFfAcvTn6gckraHyQbV64t1eQdfXCO051TpP5HYBOAdMfA2MjWNdJdf9/eOG51d92Qb0GwlWXvzwdb79/h0htL/93nwk8eYx1IvIaC9mGDvcP3pHttXgB+oA/72G80TNAW+B1/w9gHtA34L4P+PdbA1wW4bo+AHYAi/2fd/z55wDL/Bf9MmByhOt6FFjhP/8cYEDAfW/zt+N64NZI1uXf/jHwWJP7hXt7vQpsB6rw9plOBr4FfMtfbsDTft3LgPwIba/m6poGlAa8vgr8+X39bbXE/39+IMJ13RXw+vqUgDelI70GIlWXv84kvJM7Au8X7u01Bu+YwdKA/6vLo/EaUxsGEZEkEi/79EVEJAQU+iIiSUShLyKSRBT6IiJJRKEvIpJEFPoiIklEoS8ikkT+PyygkT89y/xpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f32078ae390>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title('Accuracy')\n",
    "plt.plot(history.history['accuracy'], label='train')\n",
    "plt.plot(history.history['val_accuracy'], label='test')\n",
    "plt.legend()\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 模型的评估 \n",
    "\n",
    "接下来我们通过画混淆矩阵和求F1分数来评估我们模型的表现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = model.predict(X_test)\n",
    "\n",
    "pred_cat= y_pred[:,:10]\n",
    "pred_lbl = y_pred[:,-2:]\n",
    "y_pred_cat = pred_cat.argmax(axis = 1)\n",
    "y_pred_label = pred_lbl.argmax(axis = 1)\n",
    "\n",
    "Y_test_cat = Y_test[:,:10]\n",
    "Y_test_label = Y_test[:,-2:]\n",
    "\n",
    "Y_test_cat = Y_test_cat.argmax(axis = 1)\n",
    "Y_test_label = Y_test_label.argmax(axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc\n"
     ]
    }
   ],
   "source": [
    "print(matplotlib.matplotlib_fname())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/matplotlib/font_manager.py:1320: UserWarning: findfont: Font family ['SimHei'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5,52,'预测结果')"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAHnCAYAAACxAPesAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd4VGX+9/H3TCAhjSYQIIhohCO6KKg0aUpREVgVsLB2Xd3fVh7LCoquvbALiwUFwUpRUENT6UGaCKEkdE4giKyBgBqEhBBImeePRARm0mAy95zh89prrmXOZGY+Gc9MvvO97/scl8fjQURERCQUuU0HEBEREakqKnREREQkZKnQERERkZClQkdERERClgodERERCVkqdERERCRkVTMd4HT8+7w7HbUm/qnMxaYjVFqRDjtQ5VymA5wG7RVVT/uFlKbgWEZAd4/8n3b6/T9t9XoXBHwXV0dHREREQpYjOzoiIiJSxYoKTSfwC3V0REREJGSpoyMiIiLePEWmE/iFOjoiIiISstTREREREW9FodHRUaEjIiIiXjwauhIREREJburoiIiIiLcQGbpSR0dERERCljo6IiIi4i1E5uio0BERERFvOjKyiIiISHBTR0dERES8hcjQlTo6IiIiErLU0RERERFvWl7uPGER1blr5nPcM/sl7pv/CpfefvVvN7pc3DnjWfq9+dfjm26fMowHkoZzz+yXuGf2S4EPXI5/PvZXUtYtJGXdQvr27WU6TplatEhg6eIZpKYkkbxqLl27dDAdqVwRERF8+82XrFk9n9SUJB64/w+mI1VIZGQNdmxfxUsvPWE6SoX0uaEnmzctZevm5dx/3yDTcSrEiZmdtF848b3nxMzl8XiK/H4x4azq6BQezWfK7S+Tf+Qo1SMjuG/+K9izkzl6KJdLb+vGoR9+8rrP9IdeIyt9r4G0ZWv1u5bceOP1tGvfm4iIcObNncq8eYvJz883Hc2nvLw8/vjQo6SlpWNZCcyY9hEtL+lsOlaZjh49So9et5Cbe4SoqEhS1yXxeeKXHDx4yHS0Mj3xxGBWJa8zHaNCwsLCGD78aXr0HEh2dg6rVs5hxsy5ZGUdMB2tVE7MDM7aL5z43nNi5rPFWdXRAcg/chSAsPBquFwuXC4XNWpF0/L3HUn95GvD6SrOshJYuWot+fn55OQcZteu3Vx1VVvTsUq1e3cGaWnpANh2OjVrxhhOVDG5uUeA4m9rbrcbtzu43zIXXng+lpXAvLnO2JfbtW3Dpk3byMzcz+HDucyZnUSvXl1NxyqTEzM7bb8A5733wJmZy1RU5P+LAUY6OpZl1QciS7k5x7btrKp67moR1blzxrPUbhbHklemkHfwML1evJcVb8zw+fO/H/03PEUeNkxdTMqEhVUVq9K2bElj6NB/EBMTTWRkDTp2bMtXXwVPvrL063ctKSkbTceokBo1avDN8i+4MKEZQ594kQMHfjEdqUzDhz/N0KEv0rHDlaajVEijxnFkZGQev/5Dxl7iGzc0mKh8TszstP0CnPfeA2dmPhuYKjdHA62BNj4ur1XlExcczefD3sN4p9PDXNS3A41aJ1CjVjT/W7nV62e//MdbfNh7GFMGvczFN3fi3A4tqzJapWzZmsa7701mUVIiH374BsnJ6ygoKDAdq1xNm8Yz/NWnGfzw06ajVEheXh5XXNmLhObtufWW39OgQT3TkUrVr9+1bE/byfbtO01HkSDi1P3CSe+9Xzkxc5k8Rf6/GGBqjs4x27Zn+brBsqyBgQhwJCub/Vu+p1nXVjRpZ/Gn5aMIi6hOeEwNer14Lwue+pCc/cXV+NFDuaTNTqbhpef7LIhMGTv2I8aO/QiAzz97lx3pu8wGKkdsbAzTEj9g8OBhpAd51lP99FMWqes306VLBxITvzQdx6f27S/n1ltvZMCAvsTERFO9ejUOHcph+PA3TUcr1d49+4iP/60b0iS+EWtTNhhMVD6nZXbifnEiJ7z3TuXEzKHMVEfHc5q3nZHIurFE1IwCIDy6Bud2aMmO+WsZ0/4fvNP5Yb74+1ukL0xhwVMf4gpzE1mneB5JWHg1zu92KT+l/VBV0U5LvXp1AWh92SU0bBjHunXB+2HrdruZ8vFYxo+fxIKFS03HqZB69epSq1ZNAGJiounWrePxeUbB6KmnXuWChLY0b9GBIUNe4J1xE4L+j1ny6hRatWpJw4YNiI6O4oY+PZk/f4npWGVyWmYn7hdOe++BMzOXq6jQ/xcDTHV0OlmW9Qjg4uTCxgXEV9WTxjSozQ3//VPxJGS3m5SJC/lx2/98/my18OrcMnEI7mphuNwu7K+S+W5xcBUS0xI/oFatWA4ezOaPDz5sOk6Zel/fne7dOxPXsAEPPngnAD16DgzqFQmNGsXx/nuvlUwqdDF27AQ2bgyejl4oKCwsZMjQF1iUlIjb5WLEyDFBv3rJiZmdxonvPSdmLleIHBnZ5fFUWQOlVJZlTQL+QnFhc6rRtm3fVdb9/33enYEPfQaeylxsOkKlFRnYL842vnb+YKe9ouppv5DSFBzLCOjucXTr137/TxvR8pqA7+KmOjpFtm37/CpvWZbeMyIiIqbpyMhnpKyKzolfaERERCQImerorC2Zo3MqF+CMmaoiIiKhLETm6BgpdGzbrtJj5YiIiMgZ0tCViIiISHA7q07qKSIiIhXj8Zg57o2/qaMjIiIiIUsdHREREfGmycgiIiISsjQZWURERCS4qaMjIiIi3kJk6EodHREREQlZ6uiIiIiIt6LQWF6uQkdERES8aehKREREJLipoyMiIiLetLxcREREJLg5sqPz5N6vTUeolOzPBpuOUGm1bn3DdIRKc7lcpiNUSv2oWqYjVFpmzgHTEUJe7cgY0xEq7ZcjOaYjVIrHdACn0BwdERERkeDmyI6OiIiIVLEQmaOjQkdERES8hUiho6ErERERCVnq6IiIiIgXjyc0joysjo6IiIiELHV0RERExFuIzNFRoSMiIiLedBwdERERkeCmjo6IiIh4C5GhK3V0REREJGSpoyMiIiLeQmSOjgodERER8aahKxEREZHgpo6OiIiIeAuRoSt1dERERCRkqdAp0eeGnmzetJStm5dz/32DTMc5yaRlm+g/MpH+IxIZ9VXy8e1FRR7ufHMWQyYv8rrPYxOTuGv0rEDGLNff//4AqSlJrE9dxCsvDzMdx6dPPh5L5t6NTJzw1vFtX8yaSPKquaSsW8iTTw42mM63Nle0Yt6Sz49fdu1PpdVlFzNl+rss+nYmC5dPo8+N15qOWapgfu+VximZ9/y0ia+XzeDrZTMY9eaLx7e7XC7mJn3KuPf/azBd2RISmrEoKZHU1EWsTp5nOk65pk4Zx4/7NjN50tumo/hPUZH/LwZo6AoICwtj+PCn6dFzINnZOaxaOYcZM+eSlXXAdDQO5h5l6ootJD46gDC3i7tGf8H2vVk0b1SX6attGteJ8brPt2kZuF0uA2lLV6dObf785/to3bo7BQUFLF82i99dchGbNm8zHe0kY8Z+yKRJn3P77Tcf3zboD/9HTs5hwsLC+HrRNGbOnMfmIMqdsnYj13UbCEB8k0Z8/uWH7N71A688/xrrUzZxTr26zF38KUnzlpCXd9Rw2pMF83uvNE7KfCDrF67pcpPX9jvuHsj/dmcYSFRx747/L48//jyr16RSv/45puOU6+23P2DixM8YNOjm8n9YAkodHaBd2zZs2rSNzMz9HD6cy5zZSfTq1dV0LAA8Hg8FRR6OFRSSX1iEx+OhVlQEB3OPMm/9Tga0v+ikn88vLOK9Ret5sEdrQ4l9c7mgWlgYERHhhIeH43a7+TkI/zAsXbqS7Jyck7bl5BwGoHr16lSvHtzfDfredB1fzVrAwYOHWJ+yCYCff8riQNZBatWuaTidt2B+75XGiZlPVLtOLW4e0IcJH041HaVUl1xikZNzmNVrUgH48cefDScq35Kl35KdnVP+DzqJOjqnz7Ks+kBkKTfn2LadFcg8jRrHkZGRefz6Dxl7iW/cMJARSlU7ugZ3dfkd1788BXBx79WtaFArmpemfcODPdp4/fykpRvpd+WFREVUD3zYMmRl/cLrb4xnZ3oyHo+HkSPHsHfvPtOxKmzevKlc3qYV48ZNDKpuzqn63ngtTw95+aRtl7S6CHeYm32ZPxpKVbpgfu+VxkmZa9WuxcIlieQdOcrLL77GiuXJPPn0w4z8d3APr1x44fkcyctj5owJNGrUgA8/nMLbYz40HevsEyKTkU19PR0NTAZ8ja8MAO4ObJzgdSj3KCvSfmDOE7fjdsFD4+dwxQWNOHTkKG0TGrE6fe/xn9138DDfbs/gnQd7s+dAcH2zqF27Fr16dSPhwvYUFRUxb+5UvvhiPlu2ppmOViHXXXcb0dFRfDx5DBdfbLFli206kpf4Jo2oe04dUtdtOr6tZs1YXh/zMkMefs5gMjHlikt7sC9zPy2sBD7+bBz/+PNQateuyYrlyVzVuZ3peKUKCwuj01XtuOLKXmRn57AoKZHl3ySzYcMW09HEgUwVOsds2/Y5U9ayrIGBDrN3zz7i43/7RtYkvhFrUzYEOoZPK3fs4dxzahIbGQ5A24RGrNqewbrv9tH7lakcyy/k8NF8Xpr+DV0uOped+37hhlc/pbCoiAM5eQz+YAGv39fL8G8BPbp3Jj19F4cOZQOwZMkKLr+8lWMKHYDDh3NZtGg51193dVAWOn1uvJbZsxYcv16tWjXGfTSK8WMmsjY51WCy0gXze680Tsq8L3M/AGl2Otu2pNHhqrZ06HglazckUaNGBNEx0Qwf+QxDHg2uQnjvnn2sXp1KZkn+pEXLaNWqpQqdQNMBA8+I5zRvqxLJq1No1aolDRs2IDo6ihv69GT+/CWBjuFTXK0oNuzeXzxHp6CQdd/t46oWTVjw1CDmPHEbr95xDd0uPpdhN3eia8umLHz6D8x54jY++HNfLoo/JyiKHChu73dofwXh4eFUr16dTp3akbb9O9OxyhUdHUVcXH0AwsPDufbaq7HtdMOpfOt747V8MeO31Skvj3iKDambmTp5usFUZQvm915pnJK5Vu2ahIcXD2HHNWzARRe34P13J3Npy65ccWkPHrz/EebP/Troihwofo0bN44jNjaGsLAwrurYlrQgfd9J8DPV0elkWdYjFA9dnVjYuID4QIcpLCxkyNAXWJSUiNvlYsTIMUGzguKy8+Jol9CY216bjgsX3X93Hpee18B0rEpbtWodixd/w5rV8/B4PMyYMYfk5HWmY3mZMf1DrryyNdHRUaTvSOb2QQ/x2qgXiYgonkA9fcZsvpq90HRML43jG1L3nLpsSN0MgNXyQv5w90C2bLLp1r0TAH976HG22ztNxvQSzO+90jglc4sWCYx8/XkKC4vweIp4+omX+eXAQdOxKqSwsJDnnh/J0qUzAUhM/PL4xORgNWvGBNq2Lf7s2LVzDTf1v4fUkvejY4XIHB2XxxPwBgqWZU0C/oLvOTqjbdu+q6z7VwuPD3zoM5D9WfAde6U8tW59w3SESnMF2ZL68tSPqmU6QqVl5gTfH/RQUyfS+5ARwe6XI8E1J7A8jvoDcoKCYxkB/ZA7Mv1Vv79UkTcPDfgHtamOTpFt24d83WBZllP3QREREQkypgqdsio6Z30tFxERCUUhMnRlqtBZWzJH51QuYGmgw4iIiEhoMlLo2Lb9monnFRERkQoKkeXlwX08exERETEjRAodnetKREREQpY6OiIiIuLNwOFnqoI6OiIiIhKy1NERERERb5qjIyIiIhLc1NERERERbyHS0VGhIyIiIt5C5MjIGroSERGRkKWOjoiIiHgzMHRlWVYU8BkQDRwEbgVeBa4E1tm2Pbjk50aduq006uiIiIhIsLgeWGXb9tVAMjAUiLFtuwsQbllWW8uyLj91W1kPqEJHREREvHk8/r+UL53ibg5AbcADLCi5vhDoCHTwsa1UKnRERETEW1GR/y/l2w50tCxrM8VDUwXAoZLbDlJc/NT2sa1UmqMTAPF3jjcdodJ6xl1qOkKlzc9cbzpCpWTmHDAdQYLQgSM5piNUmtvlMh2hUjwhcmqDEHUP8IVt2/+xLOsxirs7NUtuqwn8AhT62FYqdXRERETEm5mOjgvIKvn3TyX/36Pk/3sCK4FvfWwrlQodERERCRYfA7dalrUYuAN4E8izLGsZUGjbdrJt2+tO3VbWA2roSkRERLwZOGCgbdu/ANedstlr+Xh5S8pPpEJHREREvHiKQmMuk4auREREJGSpoyMiIiLeQuSknuroiIiISMhSR0dERES86ezlIiIiIsFNHR0RERHxFiKrrlToiIiIiDdNRhYREREJburoiIiIiDd1dERERESCmzo6IiIi4s2jycgiIiISqjR0JSIiIhLcVOgALVoksHTxDFJTkkheNZeuXTqYjlSqyMgarN+8mH899xgAqZu+ZumKWSz5ZhZTPx9vOF2xJ8c+ydSNUxkyesjxbd0HdOftBW8zZuEYet/R+/j2wf8ezJtz3uTtBW/z15f+isvlMhG5VH1u6MnmTUvZunk59983yHScCnFaZqflBedldtJn3K/+/vcHSE1JYn3qIl55eZjpOBXitP2iXEUe/18McHkcOAZXLTzer6GbNo2nRo0apKWlY1kJzJj2ES0v6ey3x68ZEeW3xxr2r4e54ILz+P77H3j+mRGkbvqa9ldcx9Gjx/z2HAAd6jQ/7fu26tCKyJhIrrnpGob/bTg169Zk5PSR/KPPPziWd4yXJr/EK395hYM/HyQyOpIjh48A8MSYJ1g0bRGrFqw6reedn7n+tDP7EhYWxvrURfTsdQvZ2TmsWjmHrt1uIivrgF+fx5+cltlpecGZmav6M87t5y8oderU5ptvvqR16+4UFBSwfNksHnroMTZt3uaXxy+qgr97gdgvCo5lBPSbYO6IP/r9hYp67N2Af5tVRwfYvTuDtLR0AGw7nZo1Ywwn8u2ChPNo0SKBhQuWmo5Spo0rN3Ik58jx6w3PbcjutN0cyTlCYUEhdqpNu+7tAI4XOe4wN9XDqxNMhXe7tm3YtGkbmZn7OXw4lzmzk+jVq6vpWGVyWman5QVnZnbKZ9yvXC6oFhZGREQ44eHhuN1ufg7iQhKcuV+Uy1Pk/4sBRiYjW5ZVH4gs5eYc27azApnnRP36XUtKykZTT1+m518ayjNPDadd+8uPb/N4PMxZMJXCwkLefO1dZs2cazChb3t27aFZy2bUqV+HI4eP0KZzm5MKoX++/k8u73Y5KUtTWJ202mDSkzVqHEdGRubx6z9k7CW+cUODicrntMxOywvOzHyiYP6M+1VW1i+8/sZ4dqYn4/F4GDlyDHv37jMdq0xO3y980ikgzshoYDLgq4U1ALg7sHGKNW0az/BXn6bf7+8y8fRl6t2nB+k7dpG+Y9dJhU7vXreTmbmfRo3imP7lR2zcuIXvdu42mNRbzsEc3n/5fZ794FmOHjnKzi07KSwsPH77fwb/h2rVq/H//vP/aN25NSnLUgymFQldwfwZd6LatWvRq1c3Ei5sT1FREfPmTuWLL+azZWua6WjiQKYKnWO2bc/ydYNlWQMDHQYgNjaGaYkfMHjwMNLTd5mIUKYr27am/4A+3HjT9UTHRFG9WnWys3MYNWIsAHv37mPxom9o1erioCt0AL6Z/Q3fzP4GgIeeeYg9u/acdHtBfgEr5q6g47Udg6bQ2btnH/Hxv30jaxLfiLUpGwwmKp/TMjstLzgzMwT/Z9yJenTvTHr6Lg4dygZgyZIVXH55q6AudJy6X5TFo+XlZ6SsfljAe2Vut5spH49l/PhJLFgYnPNfXnh2JK1adqX1767hX8OG8/57H/PO2x8RExMNQGzNGDp1bkeavcNwUt9q1q0JQIMmDWjdqTXJSckANDyv+IPB5XLRrkc7/pf+P2MZT5W8OoVWrVrSsGEDoqOjuKFPT+bPX2I6VpmcltlpecGZmZ3wGXeiHzL20qH9FYSHh1O9enU6dWpH2vbvTMcqkxP3i7OFqY5OJ8uyHqF46OrEwsYFxAc6TO/ru9O9e2fiGjbgwQfvBKBHz4EcPHgo0FEqpX6Dekz8+C2g+INs/DuT2LbNfKHz7AfPYrW2iIiKYMKqCTx7/7Pc9rfbOK/FeeQfy2f0k6PJP5qPy+XisVGPERUTBS7YnLyZ2ZNmm45/XGFhIUOGvsCipETcLhcjRo4J6pU14LzMTssLzszstM+4VavWsXjxN6xZPQ+Px8OMGXNITl5nOlaZnLhflCtE5ugYWV5uWdYk4C/4nqMz2rbtMgeQ/b28vKr5c3l5oJzJ8nJT/L28XEQqxt/Ly6taVSwvD4RALy8//NLdfn+hoodNCPjOYqqjU2Tbts+vEpZlOXMPFBERCSWGloP7m6lCp6yKzllfDUREREJRiAxdmSp01pbM0TmVCwj+mXIiIiLiCEYKHdu2XzPxvCIiIlJBWl4uIiIiEtxMDV2JiIhIMNMcHREREQlZIbLqSkNXIiIiErLU0RERERFvITJ0pY6OiIiIhCx1dERERMRLqJy9XIWOiIiIeNPQlYiIiEhwU0dHREREvKmjIyIiIhLc1NERERERbzpgoIiIiEhwU0cnAI4VFpiOUGnzM9ebjlBp2V8OMx2hUpre8rrpCJV24EiO6Qghr5o7zHSESisoKjQdQapCiMzRUaEjIiIiXjwhUuho6EpERERCljo6IiIi4k0dHREREZHgpo6OiIiIeNO5rkRERCRkaehKREREJLipoyMiIiLe1NERERERCW7q6IiIiIgXjyc0OjoqdERERMSbhq5EREREgps6OiIiIuJNHR0RERGR4KaOjoiIiHjR2ctFREREgpwKnRJTp4zjx32bmTzpbdNRShUREc7ipTNYsfIrklfP5d77bgfgiisvI3n1XNZv/JqhT/zdcMrS9bmhJ5s3LWXr5uXcf98g03FOMunrFPq/NJH+L01k1IzlAGRl5/K3MTO5+cUJ9H9pIgdyjgDQ+5n3ufWVydz66mT+OmaGydjH7flpE18vm8HXy2Yw6s0XAXh5+DA2b/+GOQunGk5XtmDeL0rjhMwffzyWvXs3MmHC6OPb6tWry/TpH5CamsS6dQs555w6BhOWrkWLBJYunkFqShLJq+bStUsH05EqxAn7RaUUefx/McDlxHXy1cLj/R66W9eOxMbGMGjQzdxx51/8+tg1qoX77bGioiLJzT1CVFQkq1bPpfNVfZn5xUT+/Kd/YtvpLFz0OX//6xNs3myf0fPkFRzzU+JiYWFhrE9dRM9et5CdncOqlXPo2u0msrIO+O05sr8cdlr3O3g4jztHTCHxyTsJC3Nz18ipPHdHL96bt5pebZrTo/WFZB85SkS1MMKrV6P3M+8z46m7iah+ZiO/TW95/Yzuf6LNacu5pEXnk7a1bdeGo8eOMXzEv+jd8za/PM+BIzl+eZxfBWK/8LeqzlzNHeaXx+natQMxMTHcfvtN3H333wCYMOFNEhO/YubMudSsGUte3lGOHTvz93pBUeEZP8aJmjaNp0aNGqSlpWNZCcyY9hEtL+lc/h0NCsS+XHAsw+W3B6uAg3f18Pvf2loTkwL6O4A6OsctWfot2dn+/RCvCrm5xV2FiIhw3G43kVGRuFywdet2ioqK+HTKTK7v3d1wSm/t2rZh06ZtZGbu5/DhXObMTqJXr66mYwHgwUNBURHHCgrJLyjC44Fa0TXYsfdnerS+EIDYyAjCz7CwCbTVySkcyPrFdIwyBfN+URqnZF66dCU5Ob99ptWqVZOLL7aYOXMuAIcOZfulyKkKu3dnkJaWDoBtp1OzZozhROVzyn5xNjLyyW1ZVn0gspSbc2zbzgpkHiepUSOCrxdP54KE83j6qVdp1CiOPXv2Hb89Y89err66k8GEvjVqHEdGRubx6z9k7CW+cUODiX5TOzqSu665nOv/9T644N4eV5B1KJda0TV4/P3ZfP/jL3S5pBl/63sVAC5c3DvqM9wuF/f2vIJebZob/g2gVu1aLFySSN6Ro7z84musWJ5sOlKFBPN+URonZgZo1uxcDhz4hYkT36J58/OZO3cRzz47wnSscvXrdy0pKRtNxyiXU/eLsoTKZGRTX1FHA5MBXy2sAcDdgY3jHHl5R+nY4Qbq1avL5I/HsG5d8H8ABLtDuXms2Po9c56/H7fLxUNvTqP1BY3ZsCuTTx4fxHkNavPwuC9ZvHEnV7e6gA8fuYUGtWLY90sOf3pzGlaT+jStX9vo73DFpT3Yl7mfFlYCH382jms630j2oeDvUErghIWF0a5dGzp06MP27Tv57LPx9OnTi6++WmA6WqmaNo1n+KtP0+/3d5mOIg5mqtA5Ztv2LF83WJY1MNBhnOinn7LYsGELLVpcQOPGcce3xzduxN69+8q4pxl79+wjPv63bzdN4huxNmWDwUS/WWn/j3Pr1yY2MgKAti2asDcrm6b1a3Nho3MA6Pq780nL+JGrW11Ag1rFbfS42jF0uKgp9g8/Gi909mXuByDNTmfbljQuSGjG+pRNRjNVRDDvF6VxYmaAvXv3sWPHLrZuTQNg9uwkLr20ZdAWOrGxMUxL/IDBg4eRnr7LdJxyOXW/KFOIdHRMzdEp69ULjVe2CtSrV5datWIBiImJpkuXDqxfvwWAli2b43a7ueW23zNndpLJmD4lr06hVauWNGzYgOjoKG7o05P585eYjgUUFywbvtvLsfwC8gsKWZe+h/Ma1CY2MpzMA9l4PB7W7sigWVxdjhzN53Be8byG7CNHWbvjB85vWNdo/lq1axIeXh2AuIYNuOjiFny/639GM1VUMO8XpXFiZigudA4ePESTJo0A6NKlA7adbjiVb263mykfj2X8+EksWLjUdJwKcep+UaaiKrgYYKqj08myrEcoHro6sbBxAfEmAs2aMYG2bVsTHR3Frp1ruKn/PaSmbjYRpVRxDRswbvwI3C43breL8eMnsXnTNh575FkmTBxNRI0Ipnwy/YxXXFWFwsJChgx9gUVJibhdLkaMHBM0K2suO78R7Vo04bbhH+Nyueh+WQKXnt+IR2/uyuBxX1BYWETrCxrR87IL2ZN1iEfGfwlAkcfD7V0vO971MaXLboiAAAAgAElEQVRFiwRGvv48hYVFeDxFPP3Ey/xy4CAjXnuO63t3p3ad2qzfsoTHH32OeXMWGc16qmDeL0rjlMzTp3/AlVcWf6bt2LGKAQPuZ8iQF/j88/eoVq0a3367hunTZ5uO6VPv67vTvXtn4ho24MEH7wSgR8+BHDx4yHCy0jllvzgbGVleblnWJOAv+J6jM9q27TIHZKtieXlV8ufy8kDx9/LyQDjd5eWm+HN5eaD4e3m5ePPX8vJA8vfycvEt0MvLD9xytd//1tb5bHHAl5eb6ugU2bbtszS3LMtRRYyIiIgEL1OFTlkVXcCrPRERETmFoTk1/maq0FlbMkfnVC7AGTPPREREQpiOo3MGbNt+zcTzioiIyNnFWce0FxERkcAIkaErnetKREREQpY6OiIiIuLFEyIdHRU6IiIi4i1ECh0NXYmIiEjIUkdHREREvITK0JU6OiIiIhKy1NERERERb+roiIiIiAQ3dXRERETES6jM0VGhIyIiIl5MFTqWZd0N3AOEAXcAjwFXAuts2x5c8jOjTt1WGg1diYiISFCwLCse6Gbbdg/btq8G4oAY27a7AOGWZbW1LOvyU7eV9Zjq6IiIiIgXQx2d64Awy7KSgC3ANmBByW0LgY5AgY9tq0t7QBU6ARAXVcd0hEr7/tA+0xEqre6Nw01HqJSsqX83HaHSYgeOMh0h5EVVjzAdodIOHc01HUFCRxwQbtt2D8uyhgO1gPSS2w4Cl1Bc6Ow8ZVupVOiIiIiIN4/LxLMeBJaU/HsRxfNwapZcrwn8AhT62FYqzdERERERL54i/18qYAVwacm/WwMeoEfJ9Z7ASuBbH9tKpUJHREREgoJt26nAEcuyFgNtgRFAnmVZy4BC27aTbdted+q2sh5TQ1ciIiLixVNkZOgK27YfO2WT1/Lx8paUn0gdHREREQlZ6uiIiIiIFx0ZWUREREKWx8yqK7/T0JWIiIiELHV0RERExEuoDF2poyMiIiIhSx0dERER8WJqebm/qaMjIiIiIUsdHREREfHi8ZhO4B8qdERERMSLhq5EREREgpw6OiIiIuJFHR0RERGRIKdCB2jRIoGli2eQmpJE8qq5dO3SwXSkUv3pH/cyZ9mnzFn2KT2u7wrApW0uYc6yT1mUPJO/Pfag4YSl63NDTzZvWsrWzcu5/75BpuP41Lz5BSxalMiaNfNZseIrOnduT0REBMuWzWTlytmsWTOf++673XRMACYt20j/EZ/R/z+fMerLVce3FxV5uPONGQyZlARAYVERt/438fil89MfMWnZRlOxvThhvziVUzJHRtZg/ebF/Ou54pNBfzbtPZZ8M4sVybP555C/GU5XNqe8xr+aOmUcP+7bzORJb5uO4jcej/8vJmjoCsjLy+OPDz1KWlo6lpXAjGkf0fKSzqZjebEubs61fa7h993/QHhEOBOnjWVp0gqeHT6Ef/xxKOnbd/Hp7A+Y/9XXpG3dYTruScLCwhg+/Gl69BxIdnYOq1bOYcbMuWRlHTAd7SRHjx7lT3/6J9u376RFiwQSE9+jVaurue6628nNPUJUVCRr1sxn2rTZHDx4yFjOg7l5TP1mC4mPDSTM7eKuN2eyfW8WzRvVZfpqm8Z1Y4//bJjbzaePDDh+ve8rU7j64vNMxPbilP3iRE7K/Mg//8ya1anHr9939z/IyTlMWFgYcxZM4csv5rN1S5rBhL456TX+1dtvf8DEiZ8xaNDNpqP4jYauQsju3RmkpaUDYNvp1KwZYziRbwnNm5GyeiP5+QUczsnlh9176H1jL1wuF9vtnRQVFTHr89l0v7aL6ahe2rVtw6ZN28jM3M/hw7nMmZ1Er15dTcfysnt3Btu37wQgLS2d2NjifSE39wgAERHhuN0u3G6zHwAeDxQUFXGsoJD8wiI8HqgVFcHB3DzmpaYzoP1FPu+3+X8/UjMqgibn1AxwYt+csl+cyCmZL0g4jxYtEli4YOnxbTk5hwGoXr0a1aoF7/dcp7zGJ1qy9Fuys3NMxxAfVOicol+/a0lJCZ62/om2b0unQ+criY6J4px6dbi83WWc27Qx+/buP/4zmXv2E9eogcGUvjVqHEdGRubx6z9k7CW+cUODicrXt28vUlM3A1CjRgSrVs1h+/aVjBo1jgMHDhrNVju6Bnd1bcX1L31Mz+cn06NVMxrUimb0nDU82LNNqYXYgg076XXpBQFOWzon7hdOyfz8S0N5/tkRXttnfjURO30lSxavCMpuDjjnNQ51Ho/L7xcTjJT0lmWNBbYALuDEUTsX0NS27UdN5GraNJ7hrz5Nv9/fZeLpy7Xd3smUCdP4ZNa7ZP18gNQ1wVmQhYKmTeN5+eUnuemmewHIyztK+/a9qVevLp98Mpbp02ezf/9PxvIdyj3KCvsH5jz5B9wueGjcbK64oBGHjhylbUJjVqfv8Xm/hRu+450/9QlwWgm03n16kL5jF+k7dtGu/eUn3XZjn7uIjo7i/Y9ep2XL5mzdut1QSpHAMNW7jLZt+w1fN1iWNTHQYQBiY2OYlvgBgwcPIz19l4kIFTLp/U+Z9P6nAIydMJLkb1Po2fvq47c3bNyA/Zk/GkpXur179hEf/9s3sibxjVibssFgotLFxsbw6afjefjhf7Fz5/cn3fbTT1ls2LCFzp3bMW3abEMJYeX2DM6tV5PYyHAA2iY0YtX2DNZ9l0nvlz/hWH4hh4/m89K05QzrXzzfbMsPP1IrugbxJ8zfMc1J+8WvnJD5yrat6T+gDzfedD3RMVFUr1ad7OwcRo0YC8Dhw7ksWbyCntd2C8pCxwmv8dlAZy8/M2XNvQ74vGy3282Uj8cyfvwkFixcWv4dDKp7Tm0ALm5lUT+uHqu/XQdAc+sC3G43/Qb0Jmle8P0OyatTaNWqJQ0bNiA6Ooob+vRk/vwlpmN5cbvdTJr0Fu+99zFJScsAqFevLrVqFc9piYmJpkuXDqSl7TQZk7ha0Wz4fn/xHJ2CQtZ9l8lVVhMWPH0Hc54cxKt3dqfbxU2PFzkACzZ8x7VBNGwFztkvTuSEzC88O5JWLbvS+nfX8K9hw3n/vY8ZN2YCDRrUAyA8PJwePbsa349L44TX+GxQ5HH5/WJCuR0dy7KanskT2La928fmGpZl9fex3VVyCaje13ene/fOxDVswIMP3glAj54Dja6qKc07k14jtmYM2YdyePxvzwLw3NB/88Z7w4mICGfGZ7ODbsUVQGFhIUOGvsCipETcLhcjRo4JyhUU1113Nddc04m4uPo88MAfAHj00WcZMeLZkknIbsaNm8imTduM5rysWRztLmzMbaOm4QK6t2rGpefFlXmf4mGrGwITsIKcsl+cyImZAaKiIvn403dKJtS7+WLmPObNWWQ6lk9OfI1nzZhA27atiY6OYtfONdzU/57jc/zELJennIXtlmUVAt9zegXIubZtexVTlmU1ACJLuU+2bdtZZT1otfB4R51q7LyaZf8BCkbfH9pnOkKlVQ8L3lUkvmRN/bvpCJUWO3CU6Qghr2ZElOkIlXboaK7pCGeFgmMZAW0E2Bf19vvfWmvbnIA3Myryl8Fl2/Zp9bstyypthO9NYDInF0+ekusDgLtP5/lERERETlSRQudMKrrS7nvMtu1Zvm6wLGvgGTyfiIiI+IEOGHhmgmoysoiIiIQmU5MaOlmW9Qi+j6MTbyaSiIiI/MrUuan8zVSh8y3wLr4nOLcOcBYRERE5RagMXZkqdIps2/a5dtuyrBCpIUVERMS0Cq26sizrfCq/vLysnz/d20RERCQATB3gz98qUujsBk73qFK+DhYIsLZkjs6pXEDwHdZXREREHKncQse27Wb+flLbtl/z92OKiIiI/5g627i/VeQUEGdy8D6PbdtGTtIpIiIip+9sWnX1YcnldEq7uwEVOiIiImJEhVZd2bZ9/+k8uGVZ95zO/URERMSsUJmMXJEjI1fFKSBEREREqpyzTvcsIiIiAXHWTEYWERGRs0+oTEY2dVJPERERkSqnjo6IiIh4CZXJyBU9BcTprLoKjVfIDzJzs0xHOCsUFBaYjlApNQeOMh2h0pz4pnZa9z37aK7pCCIhpSKFzgSgy2k+/oTTvJ+IiIgYdNZMRrZt+94A5BARERHxu4qcAuL9M3h8j23bD5zB/UVERMSAs2mOzr3Afafx2C7gPUCFjoiIiMM4bX5baSp6CoiPTufBz7AbJCIiInJGKlLo6BQQIiIiZ5lQGbrSAQNFREQkZOmAgSIiIuLlrFleLiIiImefItMB/ERDVyIiIhKyKnoKiNM5wrELZx4xXkRE5KznCZE/4RUpdO7j9FdPLTjN+4mIiIicsYoUOs9SXOicTmlXhM53JSIi4jhFIXKAmIoUOucBzah8oeMC0isbSERERMwrOouGrjy2be8+nQe3LOt07iYiIiLiF1peLiIiIl5CZTKylpeX6HNDTzZvWsrWzcu5/75BpuP41Lz5BSxM+pzVq+fxzYov6dy5/fHbIiNrsHXbcp5/fojBhGWbOmUcP+7bzORJb5uOUimRkTXYsX0VL730hOko5WrXtg1rVs8/fjmS+z2XXXaJ6VhlSkhoxqKkRFJTF7E6eZ7pOOVy6n7spNc5IiKCb7/5kjWr55OaksQD9//BdKRytWiRwNLFM0hNSSJ51Vy6dulgOpKUUEcHCAsLY/jwp+nRcyDZ2TmsWjmHGTPnkpV1wHS0k+TlHeXP//c427fvpEWLBD77/F0uu/QaAB4f8jeSk1MNJyzb229/wMSJnzFo0M2mo1TKE08MZlXyOtMxKiR5dQpXtr0WgKZN40lamMj69ZsNpyrbu+P/y+OPP8/qNanUr3+O6Tjlcup+7KTX+ejRo/TodQu5uUeIiookdV0Snyd+ycGDh0xHK1VeXh5/fOhR0tLSsawEZkz7iJaXdDYd64zogIEhpF3bNmzatI3MzP0cPpzLnNlJ9OrV1XQsL//7Xwbbt+8EIC0tnZqxMUDxN7UWLRJYMH+xwXTlW7L0W7Kzc0zHqJQLLzwfy0pg3tyvTUeptAED+jJt2lemY5TpkksscnIOs3pNcZH+448/G05UPifux058nXNzjwDF3R23243bHdx/rnbvziAtrXj9jW2nU7NmjOFE8qvg3nMCpFHjODIyMo9f/yFjL/GNGxpMVL4+fXuRWvJN/eVXhvHMv/5tOFFoGj78aZ566lXTMU7LwAF9+fzzL0zHKNOFF57Pkbw8Zs6YQPKqufzlz/eajhSSnPg616hRg7VrFrBr52pG/ncMBw78YjpShfXrdy0pKRtNxzhjHlx+v5hQ0SMjn86qq1J/I8uyxgJbSn7mxJX6LqCpbduPnsbznTXOPTeel156gv4330efvr3YsX0nO3Z8R4cOV5iOFlL69buW7Wk72b59Jx07XGk6TqU0bRpPvXrnHP8GH6zCwsLodFU7rriyF9nZOSxKSmT5N8ls2LDFdLSQ4sTXOS8vjyuu7EW9enX5bOp4Eqd9xf79P5mOVa6mTeMZ/urT9Pv9XaajnLFQGbqqSKFzfhU8b7Rt22/4usGyrIlV8Hxl2rtnH/Hxv3VwmsQ3Ym3KhkDHqJDY2Bg+/Ww8jz7yDDt3fs8999zGwFv6cXP/PkRHR1G9ejUOZWcz4j/OmigZjNq3v5xbb72RAQP6EhMTXfzaHsph+PA3TUcrV//+fZk2PbiHraD4vbd6dSqZmfsBSFq0jFatWgb1H2AncvLr/NNPWaSu30yXLh1ITPzSdJwyxcbGMC3xAwYPHkZ6+i7TcaREuYWObdvfV8HzlnW8xYAfizF5dQqtWrWkYcMGZGfncEOfnrwShH/M3G43Eye9xXvvfUxS0jIAnnnm3zzzTPGw1Z13DqRFiwQVOX7y1FOvHh+2uvuuW7EuSnBEkQPFw1aDBw8zHaNcyatTaNw4jtjYGHJzj3BVx7ZMnzbbdKyQ47TXuV69uuTnF3Dw4CFiYqLp1q0j73/wielYZXK73Uz5eCzjx09iwcKlpuP4xdnU0akKNSzL6u9ju5ETgRYWFjJk6AssSkrE7XIxYuSYoFtxBXDtdVdz9dVXERdXnwceKF5uef11twf1SoQTzZoxgbZtWxMdHcWunWu4qf89pKYG94ogJzr33MbUq1eXteuCsyt5osLCQp57fiRLl84EIDHxy6AfbnPifuy017lRozjef++1kknILsaOncDGjVtNxypT7+u70717Z+IaNuDBB+8EoEfPgY75fA5lLo8n8CezsCyrARBZys3Ztm1nlXX/auHxjjoDR0S16qYjVNrRgnzTESotNA5tJf7mqA8LnLkfO+01dqqCYxkB3T2+ihvk9/+0ffZ9EvBd3FRH501gMie/p389cegA4G4ToURERKRYkROrbh9MFTrHbNue5esGy7IGBjqMiIiIhCZThU5QTUYWERGRk51NZy+vCp0sy3oE38fRiTcTSUREREKNqULnW+BdfM+7ax3gLCIiInKKUBleMVXoFNm27XPNnWVZofLaioiIOFaoHEfH1Lmuyhr4C41BQRERETHOVEdnbckcnVO5gNA4pKSIiIiDFblCo+9gpNCxbfs1E88rIiIiZxdTHR0REREJYqEyYdbUHB0RERGRKqeOjoiIiHgJlVVXKnRERETES6ic60pDVyIiIhKy1NERERERLybPdWVZ1sPAANu2O1uWNQq4Elhn2/bgktu9tpVGHR0REREJGpZlRVByOijLsi4HYmzb7gKEW5bV1te2sh5PhY6IiIh48VTBpYIeAD4q+XcHYEHJvxcCHUvZVioNXQXA0YJ80xHOCqFyzAfxr94N25iOUClzMlNMRxABzExGtiyrOnC1bdtvW5b1PFAb2Fly80HgEqDAx7ZSqdARERGRYHEX8PEJ1w8CNUv+XRP4BSj0sa1UGroSERERL0VVcKkAC/izZVlzKe7U1AN6lNzWE1gJfOtjW6lU6IiIiEhQsG17iG3b19m2fT2w2bbt54A8y7KWAYW2bSfbtr3u1G1lPaaGrkRERMSL6XmPtm13Lvl/r+Xj5S0pP5EKHREREfGiIyOLiIiIBDl1dERERMRLqJzUUx0dERERCVnq6IiIiIgXdXREREREgpw6OiIiIuLFEyKrrlToiIiIiBcNXYmIiIgEOXV0RERExIs6OiIiIiJBTh0dERER8WL6XFf+okJHREREvOhcVyFm6pRx/LhvM5MnvW06SoX1uaEnmzctZevm5dx/3yDTccrltLygzIEQrHnjL4hn+LR/M3rBW7z21Wv8rv3vAGjboy1jvh7LO0vGce3t1x7/+Yeee4gJayfynxkjTEUuU7C+zqVxWt4WLRJYungGqSlJJK+aS9cuHUxHkhIuj8d5zalq4fF+D92ta0diY2MYNOhm7rjzL/5+eL8LCwtjfeoieva6hezsHFatnEPXbjeRlXXAdDSfnJYXlDkQApG3d8M2p3W/+vH1CY8IJ2NnBk0SmvD0+//iz93/j7cWvMWTtz/JkZwjjPpyFI/3f5zsX7K56IqLyD+Wz/+98Gf+edNjp513TmbKad+3NNovql7TpvHUqFGDtLR0LCuBGdM+ouUlnf36HAXHMgLaYxnV9E6//619ePekgPeJ1NEpsWTpt2Rn55iOUWHt2rZh06ZtZGbu5/DhXObMTqJXr66mY5XKaXlBmQMhmPP+mPEjGTszAPgh/QciYyKx2ljssr/nwP4D5OXmsXrRGtp0LS6ktq3dRvaBbJORSxXMr7MvTssLsHt3Bmlp6QDYdjo1a8YYTiS/MjJHx7Ks+kBkKTfn2LadFcg8TtSocRwZGZnHr/+QsZf4xg0NJiqb0/KCMgeCU/K279WenZvSqdOgLj9n/nx8+897f+KchucYTFYxTnmdf+W0vKfq1+9aUlI2mo5xxkJlebmpycijgcmArxbWAODuwMYREfGtfnx97ht2P8/d8yznX3KB6TgS5Jo2jWf4q0/T7/d3mY5yxpw3scU3U4XOMdu2Z/m6wbKsgYEO40R79+wjPv63bzhN4huxNmWDwURlc1peUOZACPa8kTGRPPXu07zz9Fj2fr+XWvVqndTBOadRPdI37jCYsGKC/XU+ldPy/io2NoZpiR8wePAw0tN3mY4jJUzN0SmrUAyVIrJKJa9OoVWrljRs2IDo6Chu6NOT+fOXmI5VKqflBWUOhGDO63a7GfL2UOZOnkPKsuIJwmmpaTS7qBl1GtShRlQN2vZoy7ol6wwnLV8wv86+OC0vFO8vUz4ey/jxk1iwcKnpOH5R5PL/xQRTHZ1OlmU9QvHQ1YmFjQuINxFo1owJtG3bmujoKHbtXMNN/e8hNXWziSgVUlhYyJChL7AoKRG3y8WIkWOCekWC0/KCMgdCMOe94poruKzTZdSpX4fr7+gNwJO3PcEHL7/PK5++isvtYtrYRLJ/KZ6A/NdX/kq7Xu2JrRXLB6s+ZMxTb5O8INnkr3BcML/OvjgtL0Dv67vTvXtn4ho24MEH7wSgR8+BHDx4yHAyMbK83LKsScBf8D1HZ7Rt22UOblbF8nIRCU2nu7zclKpYXi6hIdDLy189z//Ly4d+H/jl5aY6OkW2bfsscy3LUhEjIiIifmGq0CmroguRg06LiIg4V6h0HUwVOmtL5uicygWExiwuERERBysKkVLHSKFj2/ZrJp5XREREzi46e7mIiIh4CZUjI+tcVyIiIhKy1NERERERL6ExQ0eFjoiIiPigoSsRERGRIKeOjoiIiHgxdW4qf1NHR0REREKWOjoiIiLiRQcMFBERkZAVGmWOhq5EREQkhKmjIyIiIl60vFxEREQkyKmjIyIiIl40GVkq7JzIWNMRKu3nI9mmI1Sa2+Wsgz6EucNMR6i0/MIC0xEqbU5miukIlZKz9L+mI1RaTNdHTEeQKhAaZY6GrkRERCSEqaMjIiIiXjQZWURERCTIqaMjIiIiXkJlMrI6OiIiIhKy1NERERERL6HRz1GhIyIiIj5oMrKIiIhIkFNHR0RERLx4QmTwSh0dERERCVnq6IiIiIiXUJmjo0JHREREvOg4OiIiIiJBTh0dERER8RIa/Rx1dERERCSEqaMjIiIiXjRHJ8T0uaEnmzctZevm5dx/3yDTcUr1w08bSVo2naRl0/nvmy8AcNOAG1i8YhZLvp3FG2NeJSwszHBK36ZOGceP+zYzedLbpqNU2D8f+ysp6xaSsm4hffv2Mh3Hp+bNL2DRokTWrJnPihVf0blzeyIiIli2bCYrV85mzZr53Hff7aZjlsop770TBWvm975cSv8nR9P/ydEsXrcNgI3pP9D/ydH0/edrjJ3x9W8/+8VSrnt4JN3/8W9TccsUrK9xWZyYuSxFVXAxweXxOK9iqxYe79fQYWFhrE9dRM9et5CdncOqlXPo2u0msrIO+OXxz4mM9cvjAGxMW0arFl1O2rZu89dc0+lGDv5yiPcnvcGUSdOZP/frUh6hYn4+kn1G9/elW9eOxMbGMGjQzdxx51/8/vhul8uvj9fqdy0ZM+bfXNO9PxER4cybO5Wrr7mZ/Px8vzx+mNs/BWnTpvFERESwfftOWrRIIDHxPVq1upqoqEhyc48QFRXJmjXz6dixDwcPHjqj58ovLPBL5l9V9XuvKlR15pyl/z2t+6XtzuS5D2by4bAHOJZfyIPDP+Sjpx7gnhff4/k/3sz5jetxz4vv8sx9N9L83Dg2f5dB/dqx3P7MWBa98fgZZY7p+sgZ3f9U2i98KziW4d8PuXI82OwWvxcI43d9FtDfAdTRAaBd2zZs2rSNzMz9HD6cy5zZSfTq1dV0rAoLCwsjKiqSsLAwatSowf79P5qO5NOSpd+SnZ1jOkaFWVYCK1etJT8/n5ycw+zatZurrmprOpaX3bsz2L59JwBpaenExsYAkJt7BICIiHDcbhdud8A/X8rlxPdesGb+bu9PXHbhuVSvVo3oyAji69dhfvJmwMOFTRoQ5nZzQ8dLWZpqA3DJ+fE0qFPTbOhSBOtrXBYnZi6Ppwr+Z4KROTqWZdUHIku5Oce27axA5mnUOI6MjMzj13/I2Et844aBjFBhtWrXYv6SRPKO5PHqi6+zYnkyTz7+IktXfsmx/HzmfLmQ1HWbTMcMCVu2pDF06D+IiYkmMrIGHTu25auvFpqOVaa+fXuRmroZgBo1IliyZAYJCc0YNuwVDhw4aDidNye9934VrJkT4hvw7hdLOHzkKHn5+aRu303zJg1OKmbi6tRk1ZadBlNWTLC+xmVxYuazhanJyKOByYCvr5gDgLsDG8c52l7ak32Z+2lhJTD5s3fo3vkm/nDXADq360PWzwf4YPKbXN2jM4uTlpuO6nhbtqbx7nuTWZSUyI8//Uxy8joKCvw7dONPTZvG8/LLT3LTTfcCkJd3lPbte1OvXl0++WQs06fPZv/+n8yGlCpzYZMGDLj6Su57+T3qxEZzaUIT05HE4ULlyMimhq6O2bY9y7btmade8F38VKm9e/YRH/9b5d0kvhF79u4LdIwK2Ze5H4A0O51tW7aTcGEzCvIL2Je5n/z8fBbMW8zlV1xqOGXoGDv2I9q1v54+fe6gWrUwdqTvMh3Jp9jYGD79dDwPP/wvdu78/qTbfvopiw0bttC5cztD6UrnpPfer4I58+092/PpC3/hncfvobCoiCusZuw/8Nu8rH0HDtGgjv/mDFaVYH6NS+PEzGcLU4VOWQN1AR/ES16dQqtWLWnYsAHR0VHc0Kcn8+cvCXSMctWqXZPw8OoAxDVswEUXNydz734u/t1FxMRGA9CpS3t2BukfYyeqV68uAK0vu4SGDeNYt26D4UTe3G43kya9xXvvfUxS0jKgOHetWsVDFjEx0XTp0oG0tOAbsnDKe+9EwZw569BhALZ+v5cff8nmiouaAbDjh/0UFhUx59sNdGt9kcGEFRPMr3FpnJi5PJqjc2Y6WZb1CMXdmxN/cxcQH+gwhYWFDBn6AouSEnG7XIwYOSYoZ/c3b8GrCx4AABaaSURBVJHAyNefo7CwiCJPEf964hUy9+7nnbc+ZE7SpxQVFrJ2zXpmTZ9rOqpPs2ZMoG3b1kRHR7Fr5xpu6n/P8fkkwWpa4gfUqhXLwYPZ/PHBh03H8em6667mmms6ERdXnwce+AMAjz76LCNGPFsyCdnNuHET2bRpm+Gk3pzy3jtRMGce/Npkco4cJSYyghce7A/A0Dv78M+3pnIsv4C+nS6j+blxAIybuZjPvl7DgUOH6fX/RvDHfl25rUdwdP2C+TUujRMzlydUhq6MLC+3LGsS8Bd8D1ONtm37rrLu7+/l5VXNn8vLA6UqlpdXNX8vL69q/lpeHkj+Xl4u3k53eblJ/l5eLr4Fenn5Pc0G+P1v7Ue7EgP+QW2qo1Nk27bPA3pYluWoIkZERCQUFTnwOHu+mJqjU1ZF56yv5SIiIhK0THV01pbM0TmVC1ga6DAiIiJystDo5xgqdGzbfs3E84qIiEjF6KSeIiIiIkHO1NCViIiIBDFTx73xN3V0REREJGSpoyMiIiJeQuWAgSp0RERExIsmI4uIiIgEOXV0RERExIsmI4uIiIgEOXV0RERExEuoTEZWR0dERERCljo6IiIi4sVj4OzllmW1B0ZR3FBabdv2w5Zl/RO4EfgeuNe27Xxf20p7THV0RERExEsRHr9fKuB7oLtt252BBpZldQOuKbm+AbjJsqwGp24r6wHV0REREZGgYNt25glX84FLgMUl1xcCdwCHfWz7rLTHVKETAIfzj5qOUGku0wFOQ5GBNuuZ8BQWmI4gQSim6yOmI1TayIbXmI5QKY9mfm06giOYnIxsWdalQH3glxOiHARql1wOnbKtVBq6EhERkaBhWVZdYDTwAMWFTM2Sm2pSXPj42lYqFToiIiLixVMF/yuPZVnVgEnAYyXDWKuBbiU39wRWlrKtVCp0RERExIuhyci3AG2Bf1uWtRhIAJZalrUcaA3MsG17/6nbynpAzdERERGRoGDb9ifAJ6ds/hYYfsrPDT91W2lU6IiIiIgXE8fRqQoauhIREZGQpY6OiIiIeAmVc12p0BEREREvFVkl5QQauhIREZGQpY6OiIiIeKngcvCgp46OiIiIhCx1dERERMSLlpeLiIiIBDl1dERERMRLqMzRUaHz/9u78/AqyruN49+TPSSBQKksQSyQMEakIhI2BSwQWlkERW0RrEWLtbjQorxAxbpBNbX4WrUCgguIVkF4AdkhCAFlCRBUAkxCQkBjiEpYQlhCSN4/iIhkIZDlOTPcH69cnmUy5z5zPUx+51nmiIiISAlaXi4iIiLi5dSjIyIiIiUUajKye7Rs2YKE1fPYlhTPpo1L6dqlo+lIpYqKas6K+NlsSlzKus8+5qabOgAw/KGhJG5exuYty3lu/BjDKcsXHBzE7tSNTJgw1nSUCunTuyfJ2xPYmbyO+4YOMh2nQo7lZbA5cTmbE5czZfKLpuNckBOPsTJXDd9Af3674BnuXjKBwSuep9WgmwEIrhfGrW8/zj3xcQxZ+QJBdUMBaHBdcwaveJ57EybSfsQAg8lL+vCDN/guO5n3Zr5uOoqcx+PE5WN+ARFVGrpp0wiCgoJISUnDslowb+50olvdVGX7D/ILqJL9XHllBEFBgaSmptOyZXNmfTSNX3W7jdUJ84i54dcUFBTwyZq5DH9wNMnJdqVe62RBfpVkPt+zz46mRYtfkJGxjyeeeL5K913VLdnX15fPt62iZ+yd5OYeZeOGJXTtNoCcnINVsn9PleylpK/2JXFl0+urZd9OO8bVQZlLmtjwV5f8u37BgRQcP4lfcCBDVjzP+73H0X3CUFIXbSRt6WYCwoI5ffIUp/ML+O2CZ1jx+Bsc3P0Nd859ivjR0zhgf33Rr/nY/k8uOW9ZunXtRFhYKIMG3cbgIcOrfP8ABfmZ1XXaKFWXiB5VXiCszYyv0fcA6tEBYN++TFJS0gCw7TRq1w41nKh0X32VSWpqOgApKenUDgvD4/Hg5+dHYGAAAQH++Pj4cMBLT7iRkc2wrBYsW1r1J5nq0D7merZv38X+/d+Sl3eMJYvjiY3tajqWqzjxGCtz1So4fhIA3wA/PB4PHo+Hn1lNSFu6GYD83OOczi8gpEE4Ho+HnJRMigqLsOd9SrOe1VPQX4o1CevJzT1qOkaVKqSoyn9MUKFznn79epGU9KXpGBfUp28s2z7fTk7OIV575U12pXzK7vRNzJ+3lP1Z35qOV6q4uCcZN+4F0zEqrFHjBmRm7j97/+vMLCIaNzSYqGLq1q3Dxg1LWP3J/9G1ayfTccrlxGOszFXLN9Cfu5dO4P6N/2bLlEXUblKfE4eOcst/HmbQ4vF0evwOAEIa1OXo/pyzv3c06yChDeqaii0OYmQysmVZk4EdnOm9P7fE8wBNbdt+zESupk0jiHvhSfrdeo+Jl6+wK6+MYPyEsQy8bSjh4bXpEduFaOsmCgsLWbj4PRYtXMHOnammY/5Ev369SE1JJzU1nU4d25mO42pRLTuRlZVNdHQU8+fNoF1ML44cyTUdS6RUp0+e4v3fPEFwvTD6TBlB9hd7aHh9JP/tM45D6fvpO/WvNIttS162d/ZUu5muo1M5IbZtv1LaE5ZlvVvTYQDCwkKZO+dtRox4grS0DBMRKiQsLJQPZ7/BYyOfIj19L7fd1pv0tL1n/5AlrFlPm+tbe12h06FDW+66qz8DB/YlNDQEf38/jhw5Slzcq6ajlSnrm2wiIn781NskohFbkr4wmKhisrKyAdi5M5XkHTZRkc3YstU7czvxGCtz9Tiek8t3O/ZSL7IxhzL2k5OSCcCe+CR+Ht2Ub79IJ7RhvbPbhzaqS172IVNxxUFMDV2VVybWeAnp4+PDB+9PZurUmaxYmVDTL19hPj4+zJj5Gm+9+T6r4tcCkJmZRfsObQkICMDf35/ON8awu3gejzcZN+4FmreIIaplR0aPfo4pb8zw6iIHYFNiEq1bR9Ow4RWEhNSid5+eLF++xnSscoWH1yEg4Mzk90aNGtDqGov0PfsMpyqbE4+xMled4HphBNSuBYB/SBBNOkbz3fYM8o8cI7TRmaImouPV5KR9c7aoqdcyAo+PB6t/Z9JXbjWW/XJQVFRU5T8mmOrRCbIs6/ZSHvdQfYtRynTLb7rTvftNNGh4BcOGDQGgR887OHz4SE1HKVevX9/MzTd3pkGDn3Pf/XcDcMuvf8ea1Z+xfsMiioqKWLBgGYmJ2wwndYfTp08zesxzrIqfg4/Hw78mTvLqlTUA0VdH8fqkOApPF1JYWMjjo57m4EHv/dTrxGOszFUn5IpwYl/6Ex4fDx4fH76YsZLvd31FwnPv0+/Nkfj4+ZK1OYXdixMBWP336fR+/RF8A/3ZNffTS1pxVV0WzJtBTEwbQkJqkZG+mQG338u2bcmmY1WKW4aujCwvtyzrCiC4jKdzbdvOKeM5oOqXl1e3qlpeXpOqa3l5dXJUo8BARV8FnHaMpWZUZnm5CdWxvLwm1PTy8vaNu1X5P/lN36yp8VOfqR6dV4H3+Om5vqj4/kDg9yZCiYiIyBlu+a4rU4VOvm3bC0p7wrKsO2o6jIiIiLiTqULHqyYji4iIyE858ZsTSmOq0LnRsqyRlH4dnQgzkURERMRtTBU664FplD4fs00NZxEREZHzuGXVlalCp9C27VLXbluW5Y4jKyIi4mBuGboydcHA8paXOXHVrYiIiHghUz06W4rn6JzPA3jvpYlFREQuExq6qgTbtl828boiIiJyeTHVoyMiIiJeTBcMFBEREdcq1GRkEREREe+mHh0REREpwS1DV+rREREREddSj46IiIiU4JY5Oip0REREpAQNXYmIiIh4OUf26DjtOyLyT58yHeGiObGOd1q7cOIxlurn43FaS4ZR2atNR7gox79ZazqCI7hl6Eo9OiIiIuJajuzRERERkeqlOToiIiIiXk49OiIiIlKCW+boqNARERGREjR0JSIiIuLl1KMjIiIiJRQVFZqOUCXUoyMiIiKupR4dERERKaHQJXN0VOiIiIhICUUuWXWloSsRERFxLfXoiIiISAluGbpSj46IiIi4lnp0REREpAS3zNFRoSMiIiIluOUrIDR0JSIiIq6lQuccwcFB7E7dyIQJY01HqZBRjz9E0taVJG1dSd++sabjXFCf3j1J3p7AzuR13Dd0kOk4FXIsL4PNicvZnLicKZNfNB3nggIDA1n/6UI2Jy5nW1I89993t+lIF+TEduGEzB/8dwrZ+7cz893/nH3spZee5at9SaxNWGAwWcU88sj9bEuK5/Ntq3j+H0+YjvMT096dxYAhDzJgyIN8snYDAPOXrGTAkAfpP/hPzJq3+Oy2J06eZPQz/6TfoGH0u/sBUtMzDKW+eEXV8J8JGro6x9ixI9i4aavpGBXS+tpo+vf/De073EJgYADLln7IsmWrOXXqlOlopfL19SUu7kl69LyD3NyjbNywhHnzl5KTc9B0tHIdOHCQdjG9TMeosJMnT9Ij9k6OHTtOrVrBbNsaz0dzFnL48BHT0UrlxHbhlMyvT3qHd2fOZtDvbjv72KxZ83n33dm88u8JBpNdWN264fz5z0Np06Y7BQUFrFu7gGtbXc325F2mo2Hv3kN8wmfMfvtV8vNPcf+IsbRuZTHlnf8y661XCAwMZNhf/kaPbp35Wd1wJr31PtdYkcQ99T+cPJlPQUGB6bdw2VGPTrHIyGZYVguWLf3EdJQKsawWbNi4hVOnTnH0aB4ZGfvo3DnGdKwytY+5nu3bd7F//7fk5R1jyeJ4YmO7mo7lSseOHQfO9O74+Pjg4+O9/8yd2C6ckjkhYT25uXk/eWzDhi1eV5CVxuMBP19fAgMDCAgIwMfHhwNeknvP3q+4rtXV+Pv7ExJSi4hGDZm/eAWRza4iNCQEfz8/fnmNRcJnmwBYtiqBu++4FYDAwABCQmqZjH9RioqKqvzHBCM9OpZl/RwILuPpo7Zt59RkHoC4uCcZM2Y8nTq2q+mXviQ7dqQwZsyjhIaGEBwcRKdOMSxatNJ0rDI1atyAzMz9Z+9/nZlFROOGBhNVTN26ddi4YQnHj5/g70/9k4SE9aYjXVBQUBCfrvuYyBa/YMzY8Rw8eMh0pDI5sV04MbPT5OQc4t+vTCU9bRNFRUVMnDiJrKxs07EAiGx2FW/M+IC8vGOcyM9n25fJdO0cQ0raHr4/kEOt4GDWJyYRUqsWh4/k4u/nxz9emsSXO3Zx3bXRjH70AQICAky/jcuKqaGr14D3AE8pzw0Efl+TYfr160VqSjqpqenOKXR2pjDtzfdYFT+H774/wKZNW9UlWg2iWnYiKyub6Ogo5s+bQbuYXhw5kms6VrlOnDjBDe1iqV+/HrM/nMqcuYv49tvvTccSqbDw8DrExnajRWQHCgsLWbb0Qz7+eDk7dqaYjkZk86u489ZbuPehUdQNr8MvW12Nv78fI4ffx/BRTxMcFIgV1RxfX18KCwvZs+9r/jZyOH8f9TDPvvgqs+YvYcid/U2/jQpxywUDTRU6+bZtlzobzrKsO2o6TIcObbnrrv4MHNiX0NAQ/P39OHLkKHFxr9Z0lIsyefJ0Jk+eDsBHs6exOy3DbKByZH2TTUTEj596m0Q0YkvSFwYTVcwPnyJ37kwleYdNVGQztmz1/twA33+fw7bPk+nSpSNz5iw0HadUTmwXTszsND2630RaWsbZDxVr1nxG27atvaLQARg0sB+DBvYD4NExz9I0ojHXRrek16+6APDCy5Np2qQR4XVqU6d2GJ1irgege5dOLF+9zljui+WW6+iYGrwv7+jV+JEdN+4FmreIIaplR0aPfo4pb8zw+iIHoH79egC0ua4VDRs2YKsX/wHelJhE69bRNGx4BSEhtejdpyfLl68xHatc4eF1znYxN2rUgFbXWKTv2Wc4Vfnq169HnTq1AQgNDaFbt06kpKQZTlU2J7YLJ2Z2mq8zs+jY4QYCAgLw9/fnxhvbk5K6x3Sss3KKh4N3puzmuwM5XBvd8uxjmVnZbNzyOd06d8Dj8dCuTWuSd6UCkJj0Bc2aXmks9+XKVI/OjZZljeTM0NW5hY0HiDATyXnmznmbOnXCOHw4lz8O+6vpOOU6ffo0o8c8x6r4Ofh4PPxr4iSvnxQZfXUUr0+Ko/B0IYWFhTw+6mmvnu8CZwqyt958uXgSsofJk2fw5Zc7TccqkxPbhVMyz5s3nZh2bQgJqUV6WiK33/4Hhj1wD337xFKvXjjpaYk8OuJvLFy4wnTUEjZu3Mrq1Z+yOXEZRUVFzJu3hE1etCL2kdHPkJuXR1hICOOfGAnA+In/Yfeevfj7+fPkqIcJDDzzIemxh+5n7LMvcuzECZo1bcKf7xtsMvpFccsFAz0muqYsy5oJDKf0OTqv2bZ9T3m/7x8Q4aij7/GU9ja9mxMbuNOOsvOOsNQEHweeL5wmLzPBdIRL4l+/eY02jnphUVV+msrJTa3xBm6qR6fQtu1SL+xhWZbO/yIiIoa5ZY6OqUKnvIpOH2dEREQM06qrytlSPEfnfB7AmX2KIiIi4nWMFDq2bb9s4nVFRESkYtwydOW914YXERERqSR9qaeIiIiU4MTVt6VRoSMiIiIlFLlkMrKGrkRERMS11KMjIiIiJbhl6Eo9OiIiIuJa6tERERGRErS8XERERMTLqUdHRERESnDLqisVOiIiIlKChq5EREREvJx6dERERKQEUz06lmX9L9AO2Grb9ojK7k89OiIiIuIVLMtqC4Tatt0FCLAsK6ay+1ShIyIiIiUUVcNPBXQEVhTfXgl0quz7cOTQ1an8TI/pDCIiIm5WYOZvbTiQXnz7MNCqsjtUj46IiIh4i8NA7eLbtYFDld2hCh0RERHxFuuBHsW3ewIbKrtDFToiIiLiFWzb3gqcsCxrLXDatu1Nld2nxy0XBBIRERE5n3p0RERExLVU6IiIiIhrqdARERER13LkdXSqimVZfwVuKL7r4cyFiT4rvh8IPGzbdraJbGVxWman5QXnZK5ITuA14GTxY8ds236gRkOex4mZf+CUdnEup2RWu5DqdFkXOoCvbdtDACzLCgf+aNv2v4rvDwCCTYYrg9MyOy0vOCdzRXJ+aNv2R8WP/cVY0h85MfMPnNIuzuWUzGoXUm00dCUiIiKupUJHREREXEuFjoiIiLiWCh0RERFxLRU6IiIi4lqX+6qrI5ZlzTznfoRlWW3Oub+upgNVgNMyOy0vOCdzRXIOKF75AfBdzUUrkxMz/8Ap7eJcTsmsdiHVRt91JSIiIq6loSsRERFxLRU6IiIi4loqdERERMS1VOiIiIiIa13uq65EXM2yrF8Aey6w2V7gaeDtC2w33bbtP1iW9Q5w7wW2HWrb9juWZWUAV11g22a2bWdcYBsRkUuiQkfE/TJt225S2hPFhdDq4rvv/fDlhKVs9wfg5nMeGmbb9rQytn3nvIeibNveXca2GaVHFhGpGhq6EhEREddSoSMiIiKupUJHREREXEuFjoiIiLiWCh0RERFxLRU6IiIi4loqdERERMS1VOiIiIiIa6nQEREREddSoSMiIiKupUJHREREXEuFjoiIiLiWCh0RERFxLX17uYj7RViWVVTO83uL/z/YsqzB5Ww3/ZzbUy3LmlrOtqvPuZ1qWdYFIoqIVA9PUVF55z8RERER59LQlYiIiLiWCh0RERFxLRU6IiIi4loqdERERMS1VOiIiIiIa6nQEREREddSoSMiIiKu9f/2ReDAu0RtmwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f30901a5828>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "#生成混淆矩阵\n",
    "conf_mat = confusion_matrix(Y_test_cat, y_pred_cat)\n",
    "fig, ax = plt.subplots(figsize=(10,8))\n",
    "sns.set(font='SimHei', font_scale=0.8) \n",
    "sns.heatmap(conf_mat, annot=True, fmt='d',\n",
    "            xticklabels=cat_id_df.cat.values, yticklabels=cat_id_df.cat.values)\n",
    "plt.ylabel('实际结果',fontsize=18)\n",
    "plt.xlabel('预测结果',fontsize=18)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/matplotlib/font_manager.py:1320: UserWarning: findfont: Font family ['SimHei'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5,51,'预测结果')"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHrCAYAAAAnqSKGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHbdJREFUeJzt3XmcXlV9P/DPTTKhEjYFEiQIhO0KWFegICCLiOBPqq/6o621VRBFrVhKRahV0dpal9ZiW6kVUFGpWKkCFTeCSIGyCBbLol5kr4JSCFs2SDK3f2SMQM/ESZjMM3Pv+83rvl7znOc+zz03L8J8+Zxz7qnatg0AQJ9MG3QHAAAmmgIIAOgdBRAA0DsKIACgdxRAAEDvKIAAgN5RAAEAvaMAAgB6RwEEAPTOjEF3YG09csN8j7CGAdh0j6MH3QXorYWLb6sm8nrL7r113H/XDm223YTew2gkQABA70zZBAgAWMeGVwy6B+uMBAgA6B0JEABQ1g4PugfrjAQIAOgdCRAAUDbc3QRIAQQAFLWGwAAAukMCBACUdXgITAIEAPSOBAgAKOvwHCAFEABQ5knQAADdIQECAMo6PAQmAQIAekcCBACUdXgZvAIIACjyJGgAgA6RAAEAZR0eApMAAQC9IwECAMrMAQIA6A4JEABQ1uGtMBRAAECZITAAgO6QAAEAZZbBAwB0hwQIACjr8BwgBRAAUGYIDACgOyRAAEBR23b3OUASIACgdyRAAECZSdAAQO+YBA0A0B0SIACgrMNDYBIgAKB3JEAAQNlwd5fBK4AAgDJDYAAA3SEBAgDKLIMHAOgOCRAAUGYOEABAd0iAAICyDs8BUgABAGUdLoAMgQEAvSMBAgCK2ra7T4KWAAEAvSMBAgDKOjwHSAEEAJR5DhAAQHdIgACAsg4PgUmAAIDekQABAGUdngOkAAIAygyBAQB0hwQIACjr8BCYBAgA6B0JEABQZg4QAEB3SIAAgLIOJ0AKIACgzCRoAIDukAABAGUdHgKTAAEAvSMBAgDKOjwHSAEEAJQNYAisruvfSHJykuEkVzdNc1xd1w8muXbklN9qmmZBXdevSfLWJAuS/F7TNA+V2ka7jiEwAGAyuSPJgU3T7JNkdl3Xv57k+qZp9h85FtR1PZTkzUlelOTzSd5UalvdRRRAAEBZOzz+x6/QNM3PmqZZOvJyWZIVSXau6/rSuq4/VNd1lWTHrCyKlie5MMleo7SNSgEEAEw6dV0/O8nmTdP8ICuLmxcleWqSw5JskuQXw1sPjrwutY3KHCAAoGxAy+Drun5ako8n+e0kaZpmwUj7uUmel+S8JBuNnL5Rkgeysuh5YtuoFEAAQNlgJkHPSHJmkuObpvlZXdezkixtmmZFkr2TXJ/kpiTPqut6epKDklw5StuoDIEBAJPJ4Ul2T/KRuq4vTvLsJFfXdX1Jkmck+demaZYlOS3JpUlel+STpbbVXaRq23ad3cG69MgN86dmx2GK23SPowfdBeithYtvqybyekv+5c/H/XftU37nvRN6D6ORAAEAvWMOEABQZi8wAIDukAABAGUdToAUQABAWYc3QzUEBgD0jgQIACjr8BCYBAgA6B0JEABQNkUfljwWCiAAoMwQGABAd0iAAIAyCRAAQHdIgACAsg4/CFEBBAAUtcPdXQVmCAwA6B0JEABQZhI0AEB3SIAAgLIOT4KWAAEAvSMBAgDKOrwKTAEEAJSZBA0A0B0SIACgTAIEANAdEiAAoKw1CRoA6BtDYAAA3SEBYlzc9tOf572n/HMeXrwkQzOm54QjX5WZQ0P5i0+eteqcm++8O2d95IQ8c95WOeTNJ2WD9Z+Sqko2f+rG+cd3/2FWrBjO757w4VXn33XPgrz5d16WP3j5AYO4JZiS1ltvZr41/0uZOXMoM6ZPzz/+4xk54zNfzDnnnZE5czbP0IwZ+fKXz8+HPvgPSZJTPvGhPO95v54Z06fn8suvznF/fFLaDg97sIY6/Bygaqr+i/7IDfOnZsc76q57FuSRZcsyb+6c3PaTn+VtH/xkzj/lvY97//UnfSzf/Kf3J0kOefNJOe/v35P1Zg6N+p0v+8P35dSTjslWW2y2zvvP2G26x9GD7gK/wvrrPyWLFy/J+us/JVdd/c3s88KXZ8WK4SxcuCjTp0/P/G+fnbe99Z258cYmG2wwKwsXLkqSfP7MU3LWWefk61+7cMB3wGgWLr6tmsjrLf6bN4z779r1jz99Qu9hNBIgxsWWs5+26udt587JwiVL07Ztqmrlv+cXXPGfOWiv5475+268+Y5stMFTFD+wFhYvXpJkZRo0bdq0TJs2LQ8++HCSZGhoRoaGfvmf/l8UP9OnT8/M9WZKf3g8e4GNv6qqnllV1YlVVf39yHFiVVU7D6o/jJ+Lr74+O2/3jFXFT5JccMW1eekLn//Lk6oqr33X3+b3TvzrXHDFtf/nOy644toc/NjzgTH7tV9bL1dc+fX86KbL83cfOzX33/9gkuTr3/hCbr396nznov/IjTc2q87/1KdPzq23fTeLFi7KN79x0aC6zWQ03I7/MUkMpACqqurEJF9MUiX57shRJTmrqqo/HUSfGB933bMgf/u5c/LOow5/XNv9Dy7Mr++47aq2z33guPzLX5+Yk094Yz7+hfNz593/87jvmX/F9x9fMAFjtnTpI9lrz5dl1533zate9fLMnr0ySX3Zob+XHbffM7s+q84uu+y06vyjXn9cdth+z6xYMZz9D9h7UN2GCTWoBOioJLu3bfuhtm3PHDk+lGSPkfeKqqo6uqqqa6qquub0s782YZ1lbBYuXpJjP/zJvPMNv52tn775qvb5V16bl+z1vMedO/tpmyRJ5my6SfZ6Tp0f3f6TVe/94JY7s8mGszJ39qYT03HoqHvvXZDrrvtB9t57j1VtixYtzncu+o+85OD9H3fusmXL8tWvfiuHHXbwBPeSyawdHh73Y7IYVAE0nGTLQvvTR94ratv21LZtd2vbdrc3HP7/1lnnWHMrVgzn+I9+Ov//Jfvkhc99/EjmBZdfm5e+8JcF0OKlj2TRkqVJkocXLck1N96c7eZu8bjzD37h4wsmYGw22+xp2XjjDZMkG2wwK/vuu2du+vGtmT1nZQo0c+bMvOQl++Wmm25Jksybt3WSpKqqHHLogbmpuWUwHYcJNqhJ0H+c5NtVVf04yX+PtG2dZIckxwyoTzwJl117Y757fZP7Hngo/zr/siTJp95/bBYtXpr7H1qYXXfYZtW5Cx54OH/8kdOSJMPtcF596H7ZYeunr3p//pXX5rT3/tHE3gB0xJwtZufU0/4m06ppmTatymmnnZn/uefenP3l07PezPUybVqV8877Zr7x9W+nqqqcevpHs9GGG6Sqqlx++dU5/fR/HvQtMJlMojk7421gy+CrqpqWlUNec0eafprk6rZtV4zl85bBw2BYBg+DM9HL4Bd94LXj/rt21rs+1+9l8G3bDie5clDXBwB+hQ4vg/ccIACgrMNDYPYCAwB6RwIEAJRNomXr400CBAD0jgQIACjr8BwgBRAAUNbhVWCGwACA3pEAAQBlHR4CkwABAL0jAQIAiibT7u3jTQEEAJQZAgMA6A4JEABQJgECAOgOCRAAUOZBiAAA3SEBAgDKOjwHSAEEABS1HS6ADIEBAL0jAQIAyiRAAADdIQECAMrsBQYA9I4hMACA7pAAAQBlEiAAgO6QAAEARW3b3QRIAQQAlBkCAwDoDgkQAFAmAQIA6A4JEABQZDd4AIAOkQABAGUdToAUQABAWXf3QlUAAQCTR13Xv5Hk5Kwsv65umua4uq7fkeQVSe5IckTTNMvG2jbadcwBAgCK2uF23I8xuCPJgU3T7JNkdl3X+yU5YOT1dUleWdf17LG0re4iEiAAYNJomuZnj3m5LMmuSS4eeX1hktckWTTGtrNHu44CCAAoG+Ak6Lqun51k8yQP5JezkR5MssnI8dAY2kZlCAwAKBteB8cY1HX9tCQfT3JUVhYzG428tVFWFkRjbRuVAggAmDTqup6R5Mwkx48Mh12dZL+Rtw9KcuUatI1KAQQAFA1oEvThSXZP8pG6ri9Osn2SS+q6vizJc5Oc2zTNPWNpW91Fqradmg85euSG+VOz4zDFbbrH0YPuAvTWwsW3VRN5vfsP33/cf9c+9eyLJ/QeRmMSNABQ5kGIAEDf2AwVAKBDJEAAQFmHh8AkQABA70iAAICitsMJkAIIACjrcAFkCAwA6B0JEABQ1OUhMAkQANA7EiAAoEwCBADQHRIgAKCoy3OAFEAAQFGXCyBDYABA70iAAIAiCRAAQIdIgACAsrYadA/WGQUQAFBkCAwAoEMkQABAUTvc3SEwCRAA0DsSIACgqMtzgBRAAEBR2+FVYIbAAIDekQABAEVdHgKTAAEAvSMBAgCKLIMHAOgQCRAAUNS2g+7BuqMAAgCKDIEBAHSIBAgAKJIAAQB0iAQIACgyCRoA6B1DYAAAHSIBAgCK7AYPANAhEiAAoKjLu8ErgACAomFDYAAA3fErE6CqqrZ+Mhdo2/bOJ/N5AGAwujwJeixDYLcluSPJ2vwpPGOM1wAAmDBjKU6qtm23W5svr6qqw9OnAKDb+v4gxCfzIOwOP0QbAJiqDE8BAEX2AgMAeqfvQ2AAAJ0iAQIAirr8IMQxrQKrqmpe1nwZfHf/1ACAKW0sBdCdSS5ay+/3EEQAmKJ6/SDEtm23nYB+AACTTK9XgVVV9don8f1t27affxKfBwAYd2MZAjtj5FibHOy1SRRAADAF9X0SdNq2ff3afHlVVa9bm88BAKxLYymAbIUBAD3U60nQAEA/dXkStCdBAwC9IwECAIp6Pwl6Mpr1/CMG3QXopSV3XTroLgA8aWPdCmNtVoF1t2wEgB7o+yTozyXZdy2//3Nr+TkAgHVmLFthHDEB/QAAJplezwGqqurTT+L727Ztj3oSnwcABqTDq+DHNAR2RJIj1+K7qySfSqIAAgAmlbFuhfHZtfnyJ5keAQAD1OUhsLE8CNFWGABAp0zZ5wABAOtW35fBAwA9NDzoDqxD9gIDAHpHAgQAFLUd3tRhrFthrM0TnavYDgMAmITGUgAdmbVfzTV/LT8HAAzY8ADWctd1vWWS85PskmSDJFsluSrJD5M82jTNwSPnvSPJK5LckeSIpmmWldpGu85YCqD3ZWUBtDZpznDsBwYAU9LwYAZyFiR5cZJzHtM2v2ma3//Fi7quZyc5oGmafeq6PjHJK+u6/vcntiU5e7SLjKUA2ibJtlnzAqhKcssafgYA6LGmaZYmWVrX9WObD6jr+tIkX2ma5uQkuyW5eOS9C5O8JsmiQtuTKoDatm3vXJPO/0JVmQIEAFPVJJkEfXeSnZI8kuS8uq6/nWSTJA+NvP/gyOtS26isAgMAJq2maR7JyuIndV2fn+RZWVngbDVyykZJHhilbVSeAwQAFA2vg2NN1XW94WNe7p2V02uuTrLfSNtBSa4cpW1UEiAAYNKo63ooyTeSPCfJt5JcUtf1b2ZlCnRp0zRXjZx3SV3XlyW5M8nHmqZ59Iltq7tO1barX+NWVdWKtm2nr81NPJnP/iozZs610SoMwJK7Lh10F6C3hjbbbkIn5Vww53fH/XftwT//4qSYWCQBAgCKurwX2FifBL02q8AmRYUHAPBEYymA5q3zXgAAk06vE6C2be+YiI4AAEwUc4AAgKJJ8iDEdUIBBAAUDXe3/vEgRACgfyRAAEDRgHaDnxASIACgdyRAAEBRl7dcUAABAEVdfg6QITAAoHckQABA0XBlEjQAQGdIgACAoi5PgpYAAQC9IwECAIq6vApMAQQAFNkLDACgQyRAAECRvcAAADpEAgQAFHV5GbwCCAAoMgkaAKBDJEAAQFGXnwMkAQIAekcCBAAUmQQNAPSOSdAAAB0iAQIAikyCBgDoEAkQAFAkAQIA6BAJEABQ1HZ4FZgCCAAoMgQGANAhEiAAoEgCBADQIRIgAKDIXmAAQO/YCwwAoEMkQABAkUnQAAAdIgECAIq6nAApgACAoi6vAjMEBgD0jgQIACiyDB4AoEMkQABAUZcnQUuAAIDekQABAEVdXgWmAAIAioY7XAIZAgMAekcCBAAUmQQNANAhEiAAoKi7M4AUQADAKAyBAQB0iAQIACiyFxgAQIdIgACAoi4/CFEBBAAUdbf8MQQGAPSQBAgAKLIMHgCgQyRAAECRSdAAQO90t/wxBAYA9JAECAAoMgkaAKBDJEAAQFGXJ0FLgACA3pEAAQBF3c1/FEAAwCgGMQm6rustk5yfZJckGzRNs7yu65OT7JbkP5umOXbkvDG1jcYQGAAwmSxI8uIkVyZJXdfPz8pCaN8kM+u63n2sbau7iAQIAChqBzAI1jTN0iRL67r+RdOeSeaP/Hxhkr2SLB9j29WjXUcCBABMZpskeWjk5wdHXo+1bVQSIACgaJI8CPHBJBuN/LxRkgeSrBhj26gkQABA0XDacT/WwhVZOScoSQ7KyrlBY20blQIIAJg06roequv6wiTPSfKtJENZOSfo0iQrmqb5btM0/zmWttVdp2rbqbnKf8bMuVOz4zDFLbnr0kF3AXpraLPtqom83lu2/e1x/137idu/NKH3MBoJEADQOyZBAwBFXd4LTAEEABRNklVg64QCiHG3007b5/RTP5qNNt4wjz66LMcf/75ccumVWbLo9tz4g5uSJN/73n/lTW9+R5Lka189M3O2mJ2hoRk5++x/y19+4GOD7D5MKbfd8ZO854MnZ+HCRRkampET/ujo7P68Z+f0z38p53/roiTJsW86Igfsu2eS5E1/8u7ce9/9Wb58eQ558Yvylte/JklyxDEn5L4FD2Tm0FCS5MufPWUwNwQTRAHEuFu6dGnecPTbc9NNt6Sut8+5X/lsdt51n9x33/3ZbfeD/8/5v/PqN2XhwkWZPn16Lrn43Jx73jdzww0/GkDPYepZb72h/MU7j8u8bbbKrXf8d4454X05+QPvzrcvuTxnf+Yf8uijy3LUse/MPnu+IENDQ/nbv/izzJq1fpYvX5HXvuXtefF+L8xO289LkvzdB9+T7bZ5xoDviMlkEE+CnigKIMbdnXf+dNXPTXNLNtpog9Wev3DhoiTJ0NBQZgz5VxLWxJZbzFn187ytt8rCRYtz6+135jm7PjNDQ0MZGhrK3KdvkWuv+0H2eMFzMmvW+kmS5SuWZ/mKFYPqNgzcpFsFVlXVkYPuA+PnsMMOzrXXXp8keepTN85VV34j//6dc7Lfi/Z63HkXXnB27vrJ93PRRZdKf2AtfeeyK7NLvUN23G7bfPfa67Jo0eLcd/8D+f71N+bn99636rwjjzkx+7381dlzt+euSn+S5PiTPpjDjzwmZ335q4PoPpPQ8Do4JovJ+L/bf57kM6U3qqo6OsnRSVJN3zjTps2ayH6xhrbeem4+/KH35LDf/IMkyQ477ZW77/55dt55x/zbuZ/LC3Y/OA899HCS5KCDD8+sWevni1/4p+y6a50bb2wG2XWYcu762c/z0VM+lU/8zfuz9VZb5vDfPDSve+s78tRNNs6zd31mpk//5f/vfubjH87ixUvy9vf8VX586+3Zcbtt85H3npjZm2+aBx96OH94/EnZft422eP5zx7gHcG6NZAEqKqq60Y5rk8yZ7TPtW17atu2u7Vtu5viZ3LbcMMN8pUvfybHHvuu3HLL7UmSu+/+eZLkhz/8cW78QZMdd5j3uM8sWrQ4377oshzy0gMmurswpS1ctChvO/H9+bPj3pKtt9oySfLqVx2Wfz3jlJz2sb/KihXD2Xrulo/7zPrrPyV77v68XHrFNUmS2ZtvmiTZeKMN85ID9smNP7ppYm+CSaldB/9MFoMaApuT5LVJDisc963mc0wB06ZNyxe/8E857bQzM//CS5Ikm2yycWbOnJkkefrT52TXXercetudmTVr/cyZs3mSZObMmXnpwfunaW4ZWN9hqlmxYkXe/p4P5vBXHpq9f+MFq9oX3L9yH8gf3nRz/ue+BXnWzjtl8eIlufe+BUmSRx99NJddeU2222arLF++Ivc/8OBj2r+XHeZtM/E3w6RjCGz8nZ9kg7Ztv//EN6qqunjiu8N4OvSQA3Pggftkzhaz88Y3/n6S5G1v+7N84hMfzooVwxkeHs7b3/G+3H//A5k9e7Oce84ZWW+9mZk2bVrOOefrOf9r8wd8BzB1XHrlNbnqmu/n3vvuz9nnfiPJyiGut53453l40aJsOGtW/vJdf5IkWbx0ad56wvvy6LJlaYfbHLT/3tl/nz2zeMnSvOlP3p1ly5enHW5z8IH7Zt+9dh/kbcE6Zy8wYI3YCwwGZ6L3AvuDbX5r3H/Xfv6Or9gLDABgECbjKjAAYBLo8lCLAggAKOryZqiGwACA3pEAAQBFk+m5PeNNAgQA9I4ECAAomkwPLhxvCiAAoMgkaACADpEAAQBFJkEDAHSIBAgAKOryJGgJEADQOxIgAKCobbs7B0gBBAAUWQYPANAhEiAAoMgkaACADpEAAQBFXX4QogIIACgyCRoAoEMkQABAUZefAyQBAgB6RwIEABR1eRm8AggAKOryKjBDYABA70iAAIAiy+ABADpEAgQAFFkGDwDQIRIgAKCoy3OAFEAAQJFl8AAAHSIBAgCKhk2CBgDoDgkQAFDU3fxHAQQAjKLLq8AMgQEAvSMBAgCKJEAAAB0iAQIAirq8F5gCCAAoMgQGANAhEiAAoMheYAAAHSIBAgCKujwJWgIEAPSOBAgAKOryKjAFEABQZAgMAKBDJEAAQFGXh8AkQABA70iAAICiLj8IUQEEABQNmwQNANAdEiAAoKjLQ2ASIACgdyRAAEBRl+cAKYAAgCJDYAAAHSIBAgCKDIEBAEyAuq63TXJVkh8mebRpmoPrun5HklckuSPJEU3TLCu1rcl1DIEBAEXtOvhnjOY3TbP/SPEzO8kBTdPsk+S6JK8sta3pvUmAAIDJ5oC6ri9N8pUkTZKLR9ovTPKaJIsKbWevyQUUQABA0YDmAN2dZKckjyQ5L8mGSe4Zee/BJJuMHA89oW2NKIAAgKJBLINvmuaRrCx+Utf1+VlZ6MwdeXujJA9kZdGz1RPa1og5QADApFHX9YaPebl3kpuT7Dfy+qAkVya5utC2RhRAAEBR2w6P+zEG+9Z1/b26ri9P8tOmaa5Kckld15cleW6Sc5umueeJbWt6b1U7Rdf4z5g5d2p2HKa4JXddOuguQG8NbbZdNZHXm7fpc8b9d+1t9/3XhN7DaMwBAgCKhju8FYYCCAAomqqjRGNhDhAA0DsSIACgqMtDYBIgAKB3JEAAQFGX5wApgACAogFthTEhDIEBAL0jAQIAigaxF9hEkQABAL0jAQIAiro8CVoCBAD0jgQIACjq8oMQFUAAQJEhMACADpEAAQBFHoQIANAhEiAAoKjLc4AUQABAUZdXgRkCAwB6RwIEABR1eQhMAgQA9I4ECAAo6vIyeAUQAFDUmgQNANAdEiAAoKjLQ2ASIACgdyRAAECRZfAAAB0iAQIAirq8CkwBBAAUGQIDAOgQCRAAUCQBAgDoEAkQAFDU3fwnqbocbzF5VVV1dNu2pw66H9A3/u7BSobAGJSjB90B6Cl/9yAKIACghxRAAEDvKIAYFHMQYDD83YOYBA0A9JAECADoHQUQANA7CiAmVFVVh1RV1VRVdXNVVX866P5AX1RV9emqqu6pquqGQfcFJgMFEBOmqqrpSU5JcmiSXZK8uqqqXQbbK+iNM5IcMuhOwGShAGIi7ZHk5rZtb23b9tEkX0zyigH3CXqhbdtLkiwYdD9gslAAMZHmJvnvx7z+yUgbAEwoBRAA0DsKICbST5M84zGvtxppA4AJpQBiIl2dZMeqquZVVTUzye8m+bcB9wmAHlIAMWHatl2e5Jgk30rywyRfatv2xsH2CvqhqqqzklyRpK6q6idVVR016D7BINkKAwDoHQkQANA7CiAAoHcUQABA7yiAAIDeUQABAL2jAAIAekcBBB1XVdW2VVW1v+K4vaqqI8Zw3hkj33nGGM49YuTc28dw7raD+xMC+kgBBP3w07Ztq9KRZN5jzvvn1Zx35BO+842rOfezTzh3x9Wce8c6vG+AIgUQANA7CiAAoHcUQABA7yiAAIDeUQABAL2jAAIAekcBBAD0jgIIAOgdBRAA0DsKIACgdxRAAEDvKIAAgN5RAAEAvaMAgn6YW1VVWzqS3PaY816zmvM+84TvPG01577uCef+eDXnbrMO7xugqGrbdtB9AACYUBIgAKB3FEAAQO8ogACA3lEAAQC9owACAHpHAQQA9M7/Apv7VW034PB6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f30c4235208>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "#生成混淆矩阵\n",
    "conf_mat = confusion_matrix(Y_test_label, y_pred_label)\n",
    "fig, ax = plt.subplots(figsize=(10,8))\n",
    "sns.heatmap(conf_mat, annot=True, fmt='d',\n",
    "            xticklabels=[0,1] , yticklabels= ['0','1'])\n",
    "plt.ylabel('实际结果',fontsize=18)\n",
    "plt.xlabel('预测结果',fontsize=18)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.8853137942019752\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "          书籍       0.97      0.92      0.95       373\n",
      "          平板       0.74      0.85      0.79       998\n",
      "          手机       0.92      0.77      0.84       250\n",
      "          水果       0.94      0.90      0.92       965\n",
      "         洗发水       0.84      0.84      0.84      1006\n",
      "         热水器       1.00      0.11      0.20        54\n",
      "          蒙牛       1.00      0.99      0.99       204\n",
      "          衣服       0.87      0.91      0.89       992\n",
      "         计算机       0.92      0.90      0.91       400\n",
      "          酒店       0.99      0.96      0.98      1036\n",
      "\n",
      "    accuracy                           0.89      6278\n",
      "   macro avg       0.92      0.81      0.83      6278\n",
      "weighted avg       0.89      0.89      0.88      6278\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from  sklearn.metrics import classification_report\n",
    "\n",
    "print('accuracy %s' % accuracy_score(y_pred_cat, Y_test_cat))\n",
    "print(classification_report(Y_test_cat, y_pred_cat, target_names=cat_id_df['cat'].values))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.9066581713921631\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.92      0.89      0.90      3090\n",
      "           1       0.90      0.92      0.91      3188\n",
      "\n",
      "    accuracy                           0.91      6278\n",
      "   macro avg       0.91      0.91      0.91      6278\n",
      "weighted avg       0.91      0.91      0.91      6278\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('accuracy %s' % accuracy_score(y_pred_label, Y_test_label))\n",
    "print(classification_report(Y_test_label, y_pred_label, target_names=['0','1']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 自定义预测\n",
    "\n",
    "首先我们定义一个预测函数:通过输入一段评语,来预测它的类目"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(text):\n",
    "    txt = remove_punctuation(text)\n",
    "    txt = [\" \".join([w for w in list(jb.cut(txt)) if w not in stopwords])]\n",
    "    seq = tokenizer.texts_to_sequences(txt)\n",
    "    padded = pad_sequences(seq, maxlen=MAX_SEQUENCE_LENGTH)\n",
    "    pred = model.predict(padded)\n",
    "    #print('pred:', pred)\n",
    "    pred_cat= pred[:,:10]\n",
    "    pred_lbl = pred[:,-2:]\n",
    "    #cat_id= pred.argmax(axis=1)[0]\n",
    "    cat_id= pred_cat.argmax(axis=1)[0]\n",
    "    label_id = pred_lbl.argmax(axis=1)[0]\n",
    "    cat_lbl = cat_id_df[cat_id_df.cat_id==cat_id]['cat'].values[0]\n",
    "    label_txt = ['负面','正面'][label_id]\n",
    "    return cat_lbl,label_txt\n",
    "      "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('水果', '正面')"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('苹果好吃又不贵，已经买了很多次了')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('平板', '正面')"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('收到产品已经半个多月了，一开始用着不太好用，慢慢的就好使了，可能是习惯问题吧，主要是屏的分辨率真的不错。')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('书籍', '正面')"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('可能中外文化理解差异，可能与小孩子太有代沟，不觉得怎么样，还是比较喜欢水墨画一点风格的漫画书，但愿我女儿长大一点能喜欢（22个月中）')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('洗发水', '负面')"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('假的，不好用，头皮痒的要命。。。')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('酒店', '正面')"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('这是第三次点评，纯粹是为了拿积分。没什么可以多说了，服务不错。')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('酒店', '正面')"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#自定义review\n",
    "predict('房间挺大，就是价格贵了点')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('水果', '负面')"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#自定义review\n",
    "predict('酸的要死，下次不买了')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('平板', '负面')"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#自定义review\n",
    "predict('用了几天就好卡，上当了，退款')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('洗发水', '负面')"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#自定义review\n",
    "predict('用了几次发现头好痒，感觉过敏了')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('手机', '正面')"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('手写识别还可以')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('衣服', '负面')"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('T恤很好看，就是短了点')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('水果', '负面')"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict('吃起来太酸了，受不了，咬起来还行，能咬动')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
